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EXECUTIVE  SUMMARY 


Since  2005,  the  Environmental  Security  Technology  Certification  Program  (ESTCP)  has 
conducted  a  pilot  program  to  test  the  effectiveness  of  a  multi-technology  approach  to  UXO/MEC 
Wide  Area  Assessment  (WAA).  The  first  phase  of  this  program  was  carried  out  at  three  desert 
sites  containing  little  or  no  vegetation  and  few  non-military  land  uses;  results  from  this  first 
phase  were  very  positive.  Subsequently,  a  second  phase  of  the  pilot  program  was  added, 
including  two  new  sites:  the  Former  Camp  Beale  site  near  Marysville,  California,  and  the 
Toussaint  River  site  near  Lake  Erie.  The  Former  Camp  Beale  site  is  located  approximately  20 
miles  from  Marysville,  California,  and  covers  an  area  of  approximately  18,263  acres  (2,391 
hectares). 

URS  Corporation  and  Terra  Remote  Sensing,  Inc.  were  awarded  ESTCP  funds  to  demonstrate 
the  utility  of  high-density  lidar  and  orthophotography  as  one  component  of  the  WAA  Pilot 
Program.  The  URS  team  collected  lidar  and  orthophotography  at  two  sites  during  the  first  phase 
of  the  program,  and  at  the  Former  Camp  Beale  demonstration  site  which  is  the  subject  of  this 
report  addendum.  Data  for  the  Former  Camp  Beale  site  was  collected  over  a  two-week  period  in 
July,  2006. 

The  objectives  for  the  Former  Camp  Beale  demonstration  site  were  similar  to  those  for  the 
previous  demonstration  sites:  to  examine  the  ability  of  lidar  and  high-resolution  orthophotos  to 
identify  and  delineate  MRS  and  MEC-related  ground  features,  to  verify  or  correct  data  in  the 
existing  CSM  and  ASR,  and  to  contribute  data  that  could  be  used  to  focus  and  prioritize  the  use 
of  more  expensive  low-altitude  or  ground-based  technologies. 
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These  objectives  were  met.  Lidar  and  orthophoto  data  were  successfully  used  to  identify  MRS 
and  munitions-related  features,  and  to  verify  and  correct  the  initial  CSM.  Information  from  the 
demonstration  was  successfully  used  by  ESTCP  to  plan  subsequent  phases  of  site  investigation 
including  the  use  of  helicopter-  and  ground-based  magnetometry  and  site  reconnaissance.  The 
demonstration,  including  the  validation  activities  conducted  following  data  acquisitions, 
provided  important  insights  regarding  the  appropriate  uses  and  confidence  levels  for  both 
technologies.  All  positional  accuracy  specifications  were  met. 

The  Former  Camp  Beale  site  is  more  complex  than  the  previous  WAA  Pilot  Program  sites.  The 
area  includes  a  much  wider  variety  of  land  uses,  including  residential,  ranching  and  mining,  as 
well  as  a  wider  variety  of  munitions-related  activities.  For  this  reason,  a  much  wider  variety  of 
potential  features  appears  in  the  lidar  and  orthophoto  data.  Of  these,  some  were  clearly  MEC- 
related,  including  two  large  crater  fields,  one  bull’s  eye  aiming  target,  and  two  potential  firing 
ranges.  Significantly,  the  aiming  target  was  outside  (though  near)  any  of  the  mapped  target  areas 
in  the  initial  CSM.  Many  ground  features  were  clearly  not  ordnance-related,  including  small 
water  impoundments  and  disturbed  areas  around  houses. 

Of  the  ground  features  detected,  many  more  were  ambiguous  than  at  the  previous  demonstration 
sites.  Lidar  and  orthophotos  did  not,  by  themselves,  provide  sufficient  data  to  determine  the 
origin  of  these  features.  A  follow-up  site  visit  was  performed  in  February,  2007  to  examine 
some  of  the  ambiguous  features.  During  the  site  visit,  134  features  were  visited  over  three  days; 
the  features  were  photographed  and  examined  with  a  Schondstedt  hand-held  magnetometer.  Of 
these,  16  (12%),  were  estimated  to  be  munitions  related.  These  figures  may  not  be  representative 
of  the  entire  site,  as  right  of  entry  could  not  be  maintained  for  the  areas  of  the  site  containing  the 
bombing  targets.  Some  characteristics  were  observed  in  the  field  that  could  then  be  used  in 
subsequent  re-analysis  of  the  lidar  and  orthophoto  data  for  the  features  that  were  not  visited; 
these  included  the  shape,  size  location  in  relation  to  other  potential  features.  Other 
characteristics  were  observable  in  the  field  that  could  not  easily  be  used  in  subsequent  re¬ 
analysis,  in  particular  old  mine  shafts  were  clearly  distinguishable  as  non-MEC  in  the  field,  but 
their  shapes  were  not  easily  distinguished  from  potential  craters  in  the  lidar  data. 

Vegetation  conditions  at  the  Former  Camp  Beale  site  are  more  varied  than  at  the  earlier 
demonstration  sites,  including  areas  of  thick  grass,  brush  and  open  deciduous  forest.  The  varied 
vegetation  conditions  allowed  for  the  preliminary  examination  of  the  effects  of  vegetation  on  the 
effectiveness  of  lidar  and  orthophotography.  In  general,  the  high  lidar  data  densities  acquired 
provided  detailed  surface  models  in  the  relatively  open  forest  areas,  but  did  not  penetrate  the 
thick  brush  along  stream  channels. 

As  with  the  two  sites  previous  investigated  by  URS,  lidar  and  orthophotography  proved  to  be 
cost-effective  and  reliable  means  to  characterize  the  site,  validate  and  correct  the  initial  CSM, 
and  provide  data  to  focus  the  subsequent  application  of  subsequent  methods  of  investigation. 
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These  teehnologies  were  not  able  to  eharaeterize  every  potential  ground  feature,  but  were  useful 
for  identifying  features  for  follow-up  field  work. 
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1.0  INTRODUCTION 


1.1  BACKGROUND 

Many  millions  of  acres  of  Department  of  Defense  (DoD)  lands  are  potentially  contaminated  with 
military  munitions  or  their  components.  On  the  majority  of  these  sites,  munitions  are 
concentrated  in  specific  ranges  and  training  areas,  while  the  remainder  of  the  site  is  ordnance- 
free.  Contaminated  sites  have  traditionally  been  very  expensive  to  investigate  and  remediate 
because  of  the  nature  of  the  contamination  and  the  relatively  few  innovative  approaches 
available  to  date. 

The  current  demonstration  was  conducted  as  part  of  the  second  phase  of  the  Environmental 
Security  Technology  Certification  Program  (ESTCP)  Wide  Area  Assessment  (WAA)  Pilot 
Program.  Since  2005,  this  program  has  explored  the  use  of  an  integrated  suite  of  airborne  and 
ground-based  technologies  as  a  means  to  streamline  the  WAA  process,  resulting  in  both  cost 
savings  and  increased  confidence  in  results.  Light  Detection  and  Ranging  (lidar)  and 
orthophotography,  the  subjects  of  this  demonstration,  were  used  in  conjunction  with  synthetic 
aperture  radar  (SAR),  hyperspectral  sensing,  helicopter-based  magnetometry,  and  towed-array 
magnetometry  and  electromagnetic  induction  (EMI),  along  with  statistical  modeling,  in  an 
integrated  Geographical  Information  System  (GlS)-based  analytical  environment. 

The  first  phase  of  the  WAA  Pilot  Program  examined  three  sites,  the  Pueblo  PER  site  near  La 
Junta  Colorado,  the  Kirtland  PER  site  near  Albuquerque,  New  Mexico,  and  the  Victorville  DET 
“Y”  site  near  Victorville,  California.  All  three  sites  were  desert  bombing  ranges  with  little 
vegetation  and  few  non-military  land  uses.  The  results  of  the  first  phase  were  positive.  The 
combination  of  technologies  employed  were  successfully  used  to  locate  Munitions  Response 
Sites  (MRS)  and  munitions-related  features,  to  correct  the  initial  Conceptual  Site  Model  (CSM), 
and  to  support  regulatory  decisions  about  the  need  for  further  investigation  or  site  remediation. 
The  combination  of  technologies  employed  in  the  Pilot  Program  was  cost-effective  and  provided 
a  high  degree  of  cross-validation. 

As  a  result,  a  second  phase  was  added  to  the  program,  including  two  additional  sites.  The  first 
site.  Former  Camp  Eeale,  is  located  approximately  10  miles  from  Marysville,  California,  just  to 
the  east  of  Eeale  Air  Force  Ease.  The  site  covers  approximately  18,263  acres  (2,391  hectares). 
The  site  is  more  complex  than  the  Phase  1  sites,  with  more  vegetation  types,  more  complex 
topography,  and  a  wider  variety  of  land  uses.  The  second  site  was  the  Toussaint  River  site  near 
Lake  Erie,  which  was  primarily  an  underwater  detection  site. 


This  report  addendum  discusses  the  results  of  lidar  and  orthophoto  data  collection  and  analysis  at 
the  Former  Camp  Eeale  demonstration  site.  Overall  results  from  the  WAA  Pilot  Program,  and 
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the  relationship  of  these  two  technologies  to  the  entire  suite  of  technologies  tested,  are  discussed 
in  the  final  report  for  the  WAA  Pilot  Program  as  a  whole. 

1.2  OBJECTIVES  OE  THE  DEMONSTRATION 

The  objectives  of  the  overall  WAA  Pilot  Program  demonstration  are  unchanged  from  those 
referenced  in  the  Kirtland  AFB  Demonstration  Plan.  The  objectives  for  the  current 
demonstration  site  were  similar  to  those  for  the  previous  demonstration  sites:  to  examine  the 
ability  of  lidar  and  high-resolution  orthophotos  to  identify  and  delineate  MRS  and  MEC-related 
ground  features,  to  verify  or  correct  data  in  the  existing  CSM  and  ASR,  and  to  contribute  data 
that  could  be  used  to  focus  and  prioritize  the  use  of  more  expensive  low-altitude  or  ground-based 
technologies. 

1.3  REGULATORY  DRIVERS 

MEC  remediation  is  generally  conducted  under  authority  of  the  Comprehensive  Environmental 
Response,  Compensation,  and  Liability  Act  (CERCLA).  With  many  millions  of  acres  of  land 
potentially  contaminated  with  MEC,  estimates  of  the  cost  of  elimination  of  environmental 
liability  under  this  statute  at  known  and  former  DoD  sites  range  as  high  as  several  hundred 
billion  dollars.  These  potentially  high  costs  have  led  to  interest  in  the  development  of  innovative 
investigative  or  screening  methods,  in  order  to  reduce  the  costs  of  conducting  WAA  and 
associated  remediation  activities. 

1.4  STAKEHOLDER/END-USER  ISSUES 

The  results  from  the  Former  Camp  Beale  site,  like  those  from  the  previous  sites,  showed  that 
both  lidar  and  orthophotos  can  contribute  to  the  WAA  process  through  cost-effective  delineation 
of  MRS-  and  MEC-related  features.  The  demonstration  results  support  the  conclusion  from  the 
first  phase  sites  that  lidar  and  orthophotos  are  most  appropriately  used  during  the  early  phases  of 
the  WAA  process,  in  order  to  focus  and  prioritize  the  subsequent  application  of  more  expensive 
low-altitude  and  ground-based  technologies. 
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2.0  TECHNOLOGY  DESCRIPTION 


2.1  TECHNOLOGY  DEVELOPMENT  AND  APPLICATION 

2.1.1  Technology  Background 

Lidar  is  a  well-established  airborne  technology  for  modeling  ground  surfaces.  Topographic  lidar 
was  first  developed  in  the  late  1960s  and  early  1970s,  and  has  been  used  for  terrain  profiling 
since  the  mid-1980s.  Lidar  has  been  in  wide  commercial  use  since  around  1993,  and  the 
accuracies  and  limitations  of  lidar  for  surface  modeling  are  well  documented. 

Lidar  utilizes  the  time  of  return  for  a  laser  pulse  to  be  reflected  back  to  the  sensor  to  measure  the 
elevation  of  the  point  of  reflection.  Use  of  Global  Positioning  System  (GPS)  and  Inertial 
Measurement  Unit  (IMU)  technology  to  locate  the  sensor  precisely  in  the  air  allows  for  the 
accurate  calculation  of  the  point  of  reflection  of  the  laser  signal  from  the  ground,  buildings  or 
vegetation.  Multiple  returns  from  a  single  laser  pulse  can  be  detected,  increasing  the  chance  of 
sampling  the  ground  surface  through  gaps  in  vegetation.  Once  elevation  data  is  collected  in  the 
form  of  lidar  points,  surface  models  are  created  and  analyzed.  The  surface  modeling  process  is 
typically  conducted  using  standard  GIS  software  and  methods,  and  much  of  the  process  can  be 
successfully  automated.  Lidar  vendors  typically  guarantee  a  vertical  accuracy  of  0.15  m  and  a 
horizontal  accuracy  of  0.3-0.75  m. 


Figure  2-1 

Helicopter-mounted  Lidar  and  Orthophoto  Sensor  Equipment 
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Figure  2-2 

Lidar  System  Operations 


The  development  of  higher-speed  (50-150  kHz)  laser  scanners  beginning  around  2002  has 
significantly  improved  the  ability  of  lidar  to  locate  small  features.  Currently,  high-speed  lidar 
systems  are  being  used  to  characterize  objects  in  the  sub-meter  range,  such  as  power  line 
insulators.  The  accuracy  and  data  density  of  current  lidar  systems  suggest  that  the  technology 
could  be  used  to  detect  ground  features  indicative  of  ordnance  use,  including  targets  and  craters, 
and  that  the  presence  of  these  features  could  in  turn  be  used  to  develop  more  accurate  locations 
ofMEC. 

Digital  orthophotography  has  been  commercially  available  since  the  early  1980s,  with  steady 
improvement  in  the  resolution  (i.e.,  pixel  size)  and  precision  (i.e.,  pixel  placement)  of  the  images 
as  the  technology  of  digital  cameras,  GPS,  and  IMU  systems  has  advanced.  Since  the  mid 
1990s,  image  size  has  advanced  from  1,500  pixels  across  an  image  to  4,500  pixels.  This  has 
allowed  for  increased  flying  heights  and  a  reduced  number  of  images  for  a  given  area,  with 
consequent  cost  savings.  Commensurate  with  this  improvement  has  been  a  twofold  increase  in 
the  accuracy  of  the  IMU,  allowing  for  accurate  positioning  of  image  pixels  at  a  higher  flying 
height. 
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Airborne  digital  cameras  have  been  successfully  integrated  with  lidar  sensors.  Cameras  with  an 
image  density  of  roughly  4,000  x  4,000  pixels  are  generally  favored,  because  the  width  of  the 
images  collected  is  very  similar  to  that  of  the  typical  lidar  point  swath.  Once  collected, 
individual  digital  images  are  tiled  together  and  color  balanced,  and  the  resulting  composite  image 
is  orthorectified  using  the  lidar  data.  Orthorectification  allows  for  the  accurate  location  of  each 
photo  pixel,  eliminating  distortion  caused  by  camera  angle  and  topography.  Vendors  generally 
guarantee  a  horizontal  accuracy  of  3  pixel  widths  compared  to  ground  control  for 
orthophotography. 

Digital  images  are  collected  concurrently  with  lidar  and,  because  the  two  sensors  use  the  same 
GPS  and  IMU,  the  two  data  sets  can  be  very  accurately  integrated.  Vendors  generally  guarantee 
spatial  integration  of  orthophotos  and  lidar  within  2  pixel  widths.  Final  orthophoto  pixel  size 
depends  on  the  flight  altitude  and  the  camera  specifications;  helicopter-based  cameras  flying  at 
altitudes  of  400-450  meters  are  capable  of  pixel  sizes  of  approximately  10  cm.  Smaller  pixel 
sizes  than  this  are  generally  impractical  due  to  the  low  flight  elevations  and  slow  flight  speeds 
required  to  collect  properly  overlapping  images,  and  the  very  large  numbers  of  images  that 
would  need  to  be  combined  into  a  mosaic. 

The  ability  to  produce  spatially  accurate  orthophotos  with  relatively  small  pixel  sizes  suggests 
that  this  technology  could  be  used  to  identify  ordnance-related  features,  and  to  cross-validate 
technologies  such  as  lidar. 

2.2  PREVIOUS  TESTING  OF  THE  TECHNOLOGY 

As  part  of  the  first  phase  of  the  ESTCP  WAA  Pilot  Program,  lidar  and  orthophotos  were  tested  at 
three  demonstration  sites:  Former  Kirtland  AFB  PBR  near  Albuquerque,  NM,  Victorville  DBR 
“Y”  site  near  Victorville,  CA  and  Pueblo  PBR  near  Pueblo,  CO.  Lidar  and  orthophotos  were 
successfully  used  to  detect  large  bombing  targets  such  as  bull’s-eye  rings  and  simulated  ship 
targets,  despite  the  fact  that  the  berms  making  up  these  targets  were  weathered  and  frequently 
less  than  10  cm  tall,  and  were  not  visible  to  ground  crews  on  the  site.  Individual  munitions- 
related  features  were  also  detected  using  lidar,  including  potential  craters  down  to  approximately 
1  m  in  diameter. 

2.3  FACTORS  AFFECTING  COST  AND  PERFORMANCE 


The  following  factors  were  found  to  affect  either  cost  or  performance  at  both  the  first  phase  sites 
and  the  Former  Camp  Beale  site.  These  factors  are  consistent  with  industry  standard  guidance 
on  the  use  of  lidar  and  orthophotos  in  a  variety  of  applications. 
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Table  2-1 

Cost  and  Performance  Factors 


Item 

Cost  and  Performance  Factors 

Lidar  data  density 

Higher  lidar  data  density  performs  better  but  is  generally  more  expensive, 
due  to  the  need  to  acquire  and  process  additional  flight  lines. 

Orthophoto  data 
density 

Orthophotos  with  smaller  pixels  are  more  effective  but  more  expensive, 
because  of  the  additional  flight  lines  required  and  the  larger  number  of 
individual  digital  images  which  must  be  processed  and  mosaiced. 

Accuracy  and  precision 
requirements 

Accuracy  and  precision  are  largely  determined  by  the  equipment  used  and 
the  care  of  the  operators.  Projects  with  extremely  high  accuracy 
requirements,  such  as  creation  of  contour  lines  under  1-foot  intervals,  can 
only  be  accomplished  by  vendors  with  newer  equipment. 

Site  location  and 
logistics 

Sites  with  longer  flying  times  to  an  airport  will  be  more  expensive,  as 
they  will  require  either  longer  flight  times  or  placement  of  fuel  on  the  test 
site. 

Verification  of 
accuracy  and  precision 

Verification  of  accuracy  and  precision  is  accomplished  through 
placement  survey  control  and  comparison  of  lidar  and  orthophoto  data  to 
such  control.  Projects  with  higher  verification  requirements  will  be  more 
costly,  although  this  factor  is  small  compared  to  other  factors. 

Site  size 

Larger  sites  can  achieve  substantial  cost  savings  through  amortization  of 
fixed  cost,  such  as  mobilization  and  project  planning,  as  well  as  through 
increased  efficiency  in  data  acquisition  and  processing. 

Vegetation  conditions 

Vegetation  impacts  the  ability  of  both  lidar  and  orthophotos  to  view  or 
model  the  ground  surface.  More  densely  vegetated  sites  will  have  higher 
costs  due  to  the  requirement  for  additional  lidar  passes  to  achieve 
sufficient  point  density  at  the  ground  surface. 

Permitting  and  site 
access 

Some  DoD  sites  contain  high-security  areas,  which  can  raise  costs  for 
pre-flight  planning  and  data  collection.  These  costs  result  from 
restrictions  on  site  access,  time  to  acquire  needed  clearances,  and  longer 
flight  times  to  avoid  restricted  areas. 

Sites  with  environmental  constraints  do  not  normally  impose  higher  costs 
for  lidar  and  orthophotography,  due  to  the  airborne  nature  of  the 
technologies.  However,  the  presence  of  sensitive  species  may  affect  pre¬ 
flight  planning  and  scheduling  (and  thus  costs)  for  projects  which  require 
landing  to  re-fuel  on  the  site. 
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2.4  ADVANTAGES  AND  LIMITATIONS  OF  THE  TECHNOLOGY 

The  use  of  lidar  and  orthophotos  offers  several  advantages  eompared  to  the  traditional 
approaehes  to  MRS  investigation.  As  with  the  faetors  affeeting  the  eost  and  performanee,  these 
faetors  were  found  to  apply  to  both  the  first  phase  sites  and  the  Former  Camp  Beale  site. 


Table  2-2 

Technology  Advantages  and  Limitations 


Item 

Advantages 

Rate  of  coverage 

In  an  operational  setting,  data  collection  rates  of  5,000  acres  per  day  or 
higher  can  be  expected  for  lidar  and  orthophotos.  This  compares  favorably 
to  maximum  collection  rates  of  around  500  acres  per  day  for  helicopter- 
based  magnetometry,  and  20  acres  per  day  for  towed-array  magnetometry. 

Ability  to  delineate 

MRS  and  MEC-related 
features 

Lidar  and  orthophotography  can,  under  some  circumstances,  successfully 
reveal  MRS  and  MEC-related  surface  features  even  many  years  after  their 
last  use. 

Enhanced  planning  and 
risk  assessment 

Because  they  can  cover  entire  sites  relatively  quickly  and  at  lower  cost, 
these  technologies  can  be  used  to  locate  and  prioritize  appropriate  areas  for 
use  of  more  costly  low-altitude  and  ground-based  technologies. 

Other  benefits 

Both  technologies  provide  highly  detailed  topographic  data  that  can  be 
integrated  into  a  facility’s  CADD  or  GIS  system  and  used  in  subsequent 
phases  of  site  investigation,  site  remediation,  and  range  management 

Lidar  and  orthophotos  have  the  following  limitations  eompared  to  other  existing  teehnologies: 


Item 

Disadvantages 

MEC  detection 

Neither  lidar  nor  orthophotography  can  directly  detect  shell  casings  or 
other  MEC  components  such  as  scrap.  Consequently,  further 
investigation  with  magnetometers  or  electromagnetic  induction  (EMI) 
sensors  is  required. 

Elevation  data 

Orthophotos  do  not  contain  elevation  information.  In  practice,  it  is 
sometimes  difficult  to  distinguish  small  surface  depressions  from  small 
mounds  or  shadows  using  orthophotos  alone. 

Vegetation  effects 

Since  both  lidar  and  orthophotos  are  light-based  technologies,  neither  will 
penetrate  vegetation.  Orthophotos  do  not  “look  through”  vegetation,  and 
lidar  point  densities  will  be  lower  in  vegetated  areas.  However,  lidar  is 
frequently  successful  in  penetrating  small  openings  between  and  within 
vegetation,  and  this  success  has  increased  with  the  speed  of  lidar  sensors 
and  the  development  of  the  ability  to  measure  multiple  returns. 
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3.0  DEMONSTRATION  DESIGN 

3.1  PERFORMANCE  OBJECTIVES 

The  primary  performance  objectives  for  these  technologies  were  similar  to  those  for  the  first 
phase  sites,  as  applied  to  the  more  complex  environment  of  the  Former  Camp  Beale  site. 

•  Clarify  whether  and  to  what  extent  lidar  and  orthophotos  can  delineate  MRS 
boundaries  and  MEC-related  features,  and  contribute  to  the  focusing  and 
prioritization  of  subsequent  low-altitude  and  ground-based  work. 

•  Reveal  relationships  between  the  density  of  lidar  and  orthophoto  data,  their  levels  of 
cost,  and  their  ability  to  accurately  locate  MRS  boundaries  and  MEC-related  ground 
features. 

•  Clarify  whether  and  to  what  extent  lidar  and  orthophotos  can  verify,  reveal  errors  in, 
or  improve  the  accuracy  of  the  initial  CSM. 

•  Contribute  data  and  analysis  to  the  overall  combination  of  technologies  used  in  the 
WAA  Pilot  Program,  in  a  manner  that  is  timely  to  the  application  of  the  other 
technologies  demonstrated,  in  formats  useable  by  other  demonstrators,  and  with 
sufficient  positional  accuracy  compared  to  project  control  points  to  allow  meaningful 
coordination  and  comparison. 

Specific  performance  criteria  and  performance  metrics  related  to  each  of  these  objectives  are 
established  in  the  Technology  Demonstration  Plan  for  this  site. 

3.2  SELECTION  OF  TEST  SITE 

The  demonstration  site  was  chosen  by  the  ESTCP  Program  Office.  Details  of  the  site  selection 
process  can  be  found  in  the  final  report  for  the  second  phase  of  the  WAA  Pilot  Program. 

3.3  TEST  SITE  HISTORY,  CHARACTERISTICS  AND  PRESENT  OPERATIONS 

The  Former  Camp  Beale  site  consists  of  87,672  acres  approximately  20  miles  east  of  Marysville, 
California,  between  Yuba  and  Nevada  counties.  The  site  is  located  immediately  to  the  east  of 
Beale  Air  Force  Base.  The  demonstration  site  was  used  by  the  DoD  for  ground  ranges,  moving 
target  ranges  and  bombing  ranges  between  1943  and  1959.  Historic  photographs  revealed 
extensive  ground  disturbances,  expected  to  have  been  created  during  previous  MM-related 
activities.  Other  areas  from  the  historical  photographs  were  noted  as  disturbed,  either  by  ground 
scarring,  visible  craters,  or  other  activities.  According  to  the  CSM,  cleanup  activities  were 
conducted  in  1947,  and  1958  -  1959. 
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The  Former  Camp  Beale  site  was  excessed  and  sold  between  1959  and  1964  and  now  contains 
both  private  land  and  state  land  located  within  the  Spenceville  Wildlife  Reserve. 

Figure  3-1 

Former  Camp  Beale  Demonstration  Site  Location 


3.4  PRE-DEMONSTRATION  TESTING  AND  ANALYSIS 

Pre-demonstration  activities  included  scheduling,  flight  line  planning,  planning  for  placement  of 
project  controls  and  securing  appropriate  site  access.  Scheduling  included  reservation  of  a  local 
helicopter  and  pilot,  determining  the  most  appropriate  local  airport  from  which  to  base 
operations,  and  arranging  shipping  for  the  sensor  equipment  and  accommodations  for  the  field 
crew.  Helicopter  use  was  scheduled  to  include  an  extra  day  for  re-acquisition  of  any  problem 
areas  or  missed  areas. 

Flight  lines  were  planned  to  ensure  complete  site  coverage,  minimize  the  number  of  turns,  and 
achieve  planned  overlap.  Digital  imagery  was  planned  for  acquisition  at  periods  of  low  sun 
angle.  Previous  testing  had  shown  that  the  shadows  created  by  low  sun  angle  were  useful  in 
detecting  shallow  features.  Flight  line  planning  included  coordination  with  Beale  Air  Force  Base 
to  secure  permission  to  turn  in  the  base  air  space. 


Planned  controls  included  18  surveyed  control  points,  10  test  craters  and  4  vertical  control 
structures.  The  demonstration  site  includes  a  considerable  amount  of  private  land  currently  used 
for  ranching  and  as  rural  residences,  and  right  of  entry  could  not  be  secured  for  placement  of 
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control  points  on  some  private  land  areas.  In  these  parts  of  the  project  area  survey  control  points 
were  planned  for  placement  in  the  right-of-way  of  public  roads.  Test  craters  and  vertical  control 
structures  were  planned  for  placement  on  public  land  and  private  land  with  right  of  entry. 

3.5  TESTING  AND  EVALUATION  PLAN  AND  PROCEDURES 
3.5.1  Demonstration  Set-Up  and  Start-Up 

At  the  site,  the  lidar  and  orthophoto  sensor  system  was  installed  into  a  helicopter  owned  by  a 
local  vendor.  Use  of  local  helicopters  and  pilots  is  a  common  industry  practice  which  allows  the 
lidar  vendor  to  ship  only  the  sensor  package  rather  than  the  aircraft.  The  use  of  local  helicopter 
vendors  also  allows  for  the  use  of  local  pilots  who  have  better  knowledge  of  local  weather 
patterns  and  flight  clearance  requirements. 


Figure  3-2 


Equipment  installation 


Sensor  pod  mounted  below  helicopter,  control 
console  visible  through  window. 

Concurrently  with  installation  of  the  sensor  equipment,  the  field  crew  installed  control  points, 
test  craters,  and  vertical  control  structures.  Examples  are  shown  in  the  following  images. 
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Figure  3-3 


Control  point  locations,  Former  Camp  Beale 


Example  Camp  Beale  control  point 


Figure  3-4 


Control  point  within  road  right-of-way.  Control  point  within  road  right-of-way. 
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Figure  3-4 


Vertical  control  structure 


Existing  bunker  used  as  vertical  control  structure 

Figure  3-5 


1.5  m  test  crater. 


0.3  m  test  crater. 
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3.5.2  Period  of  Operation  and  Area  Characterized 

On  July  27,  2006,  a  calibration  flight  at  450  meters  was  conducted  to  establish  appropriate  pitch, 
yaw,  and  roll  correction  factors.  A  laser  pulse  rate  of  75  kHz  was  selected  based  on  performance 
at  a  variety  of  speeds  tested  and  the  contract  specifications. 

Data  collection  flights  were  begun  once  mobilization,  sensor  installation,  and  calibration  flights 
were  complete.  Data  collection  flights  were  conducted  from  the  airport  at  Yuba,  California. 

Data  collection  periods  were  as  follows: 


Table  3-1 

Data  Collection  Periods 


Activity 

Period 

Preflight  planning 

June  1  -  July  20  2006 

Mobilization 

July  20,  2006 

Calibration  flight 

July  27,  2006 

Data  collection 

July  22-26,  2006 

Verification  of  radar  corrections 

July  27-31,2006 

Demobilization 

July  31,  2006 

3.5.3  Site  Conditions  Affecting  Data  Collection 

Data  collection  was  affected  by  two  site  conditions  that  should  be  considered  in  planning  for 
future  use  of  these  technologies.  First,  data  collection  took  place  during  mid-summer,  and 
ambient  air  temperatures  during  data  collection  often  exceeded  1 10°  F.  These  temperatures  led 
to  difficulties  with  helicopter  operation,  including  inability  to  reach  planned  altitudes  and  speeds. 
High  temperatures  also  affected  the  sensor  equipment  directly.  As  a  result,  data  collection  could 
only  take  place  in  the  early  morning  hours.  As  a  result,  the  day  reserved  for  re-acquisition  of  any 
missed  or  problem  areas  was  used  to  complete  data  collection. 

Second,  Beale  AFB  is  the  site  of  one  of  three  installations  that  are  part  of  the  Phased  Array 
Warning  System  (PAWS)  radar  system,  designed  to  detect  and  track  sea-launched  ballistic 
missiles.  The  high-intensity  radar  signals  from  this  installation  disrupted  the  GPS  time  signal 
used  by  the  lidar  and  orthophoto  sensors,  rendering  the  data  unusable.  The  effect  was  noted 
within  several  kilometers  of  the  radar  station  and  at  altitudes  up  to  approximately  500  m.  The 
problem  was  noted  during  the  daily  QA/QC  checks  on  the  first  day  of  data  collection.  A  sample 
of  the  data  was  sent  to  the  TRSI  office  in  Sidney,  BC,  and  a  solution  was  developed  to  re-insert 
the  correct  GPS  times.  The  field  crew  stayed  on  the  site  for  an  additional  5  days  while  this 
solution  was  developed  and  tested. 
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Figure  3-6 


Temperature  during  data  collection  PAWS  radar  station,  Beale  AFB 

3.5.4  Demobilization 

Demobilization  of  the  lidar  and  orthophoto  equipment  consisted  of  dismounting  the  sensor 
system  from  the  helicopter,  packing  and  shipping.  Demobilization  required  approximately  4 
hours. 


3.6  SELECTION  OF  ANALYTICAL/TESTING  METHODS 

Analysis  of  lidar  data  is  performed  in  two  steps:  conversion  of  sensor  output  to  spatially  correct 
lidar  points,  and  then  conversion  of  these  points  to  useable  GIS  products  such  as  surface  models 
or  contour  lines.  Processing  of  the  Former  Camp  Beale  lidar  data  into  point  fdes  was 
accomplished  using  a  suite  of  software  including  TerraSolid  and  custom  algorithms  written  in 
this  software  by  TRSI.  TerraSolid  is  an  industry  standard  software  package  for  processing  lidar 
data. 

Creation  of  GIS  products  and  analysis  were  accomplished  using  ESRI’s  ArcGIS  software  suite. 
ArcGIS  was  used  because  it  is  the  most  widely  used  GIS  package  by  US  government  agencies 
and  private  contractors.  As  such,  it  is  appropriate  to  develop  analysis  methods  and  resulting 
products  that  can  be  duplicated  by  typical  federal  facilities  staff. 

Analysis  of  orthophoto  data  was  also  performed  in  two  steps:  creation  of  orthorectified  images 
from  the  large  number  of  individual  digital  images  collected  followed  by  visual  examination  of 
the  image  to  locate  potential  MRS  and  munitions-related  features.  Processing  of  the  digital 
imagery  to  create  the  orthophoto  mosaic  was  accomplished  using  software  from  TerraSolid  and 
PCI.  This  software  is  also  the  industry  standard. 
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3.7  RESULTS 

3.7.1  Data  Collection 

Data  collection  flights  took  place  on  July  22  through  26,  2006  with  a  total  of  120  flight  lines 
accomplished.  Lidar  data  was  collected  at  altitudes  of  300  and  450  meters.  Orthophoto  data  was 
collected  concurrently  with  lidar  during  the  450  meter  flight  in  order  to  produce  images  with  a  10 
cm  pixel  size.  Complete  coverage  was  obtained  for  each  flight,  and  flight  line  overlap  was 
within  specifications. 


Figure  3-7 


Former  Camp  Beale  achieved  flight  lines 


3.7.2  Safety  Issues 

No  special  safety  issues  were  encountered  during  data  collection. 

3.7.3  Data  Processing  Steps 

Processing  of  the  sensor  output  to  create  lidar  points  was  performed  by  TRSI.  Following  return 
of  the  data  to  the  office,  calibration  factors  determined  in  the  field  were  checked,  fine-tuned,  and 
applied  to  laser  range,  GPS,  and  IMU  data  to  produce  x,y,z  values  for  each  point.  Lidar  points 
were  then  transformed  into  the  delivery  datum  and  projection,  and  coded  to  indicate  returns  from 
ground  vs.  non-ground  surfaces.  Additional  fields  for  each  point  included  the  intensity  value, 
gps  date,  and  flight  line  number.  Lidar  points  were  exported  as  text  files  for  delivery  to  URS. 
Lidar  data  from  each  lidar  flight  was  processed  and  delivered  separately  in  order  to  allow  for 
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separate  analysis  of  data  from  each  flight  altitude.  Lidar  data  was  delivered  in  345  blocks  (for 
each  flight),  each  covering  1/4  square  kilometer,  in  order  to  keep  file  sizes  manageable.  Even 
with  this  size  limitation,  each  of  the  separate  lidar  data  files  contained  up  to  6.5  million 
individual  lidar  points. 

Digital  image  processing  was  done  by  TRSI.  The  procedure  included  mosaicing  of  the 
individual  digital  images  collected  during  flight,  transformation  of  the  consolidated  image  to  the 
delivery  datum  and  projection,  orthorectification  using  the  lidar  data,  color  balancing,  and 
trimming  to  the  delivery  tiles.  For  the  Former  Camp  Beale  site,  orthophotos  were  created  in 
blocks  of  2.675  square  kilometers,  in  order  to  keep  file  sizes  manageable.  Even  with  this  size 
limitation,  the  19  individual  orthophoto  blocks  were  approximately  2  GB  each. 

Figure  3-8 


Orthophoto  data  blocks. 

Conversion  of  the  random  lidar  point  data  to  GIS  products  was  conducted  by  URS,  using  the 
following  steps: 

•  The  lidar  points  were  converted  to  Arcinfo  point  shape  file  format,  with  the 
horizontal  locations  determined  by  the  northing  and  easting  values  in  the  lidar  point 
file,  and  the  elevation  value,  intensity  value,  and  the  code  for  ground  or  non-ground 
return  retained  as  attributes. 

•  Point  shape  files  were  converted  to  Triangulated  Irregular  Network  (TIN)  files.  A 
TIN  is  an  elevation-based  surface  model  where  each  point  forms  a  vertex  in  a 
network  of  linked  triangles.  TINs  were  created  separately  from  the  lidar  points  coded 
as  ground  returns  and  from  the  entire  lidar  point  set,  using  Arcinfo’ s  TIN  creation 
functions.  TINs  were  created  for  each  lidar  flight. 
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•  Digital  elevation  models  (DEMs)  were  ereated  from  the  TENs.  A  DEM  is  a  regularly 
spaeed,  gridded  array  of  elevation  values.  DEMs  were  created  using  the  each  of  the 
TEN  files  as  inputs.  DEMs  were  created  in  the  Arcinfo  GRID  file  format,  which 
allows  for  additional  analysis  that  cannot  be  performed  directly  on  the  TIN  file.  All 
DEMs  were  created  using  0.3  m  (1  ft)  grid  cell  sizes.  This  value  was  chosen  as  the 
smallest  cell  size  that  would  be  supported  by  the  lidar  data  densities  acquired. 

Automation  of  GIS  processing  and  analysis  was  accomplished  for  many  of  the  process  steps. 

Once  the  initial  GIS  products  were  created,  the  lidar  data  were  examined  to  detect  missing  data, 
spatial  discrepancies,  or  artifacts  in  the  surface  that  would  indicate  improper  calibration  or  other 
problems.  Further  data  quality  review,  including  review  for  spatial  accuracy,  was  performed 
based  on  the  parameters  given  in  Appendix  C  of  the  Technology  Demonstration  Plan.  All  data 
met  data  quality  specifications. 

Following  creation  of  initial  GIS  products  and  initial  QA/QC  review,  hillshades  were  created  for 
each  of  the  DEMs.  Hillshades  are  three-dimensional  depictions  of  the  surface  with  shadows 
formed  by  a  simulated  light  source  placed  above  the  surface  at  an  altitude  and  azimuth  chosen  by 
the  operator.  The  default  settings  for  hillshades  in  ArcGIS  Spatial  Analyst  were  used,  and  then 
varied  as  needed  during  the  analysis.  Hillshades  were  saved  in  Arcinfo  GRID  format. 

Once  the  initial  data  processing  steps  were  completed,  the  lidar  and  orthophoto  data  were 
examined  using  the  following  steps: 

•  Each  lidar  hillshade  and  orthophoto  data  set  was  visually  inspected  for  potential 
MRS. 

•  Potential  MRS  were  identified  and  drawn  as  Arcinfo  point  or  polygon  shape  files  for 
each  data  set. 

•  Each  data  set  was  visually  inspected  for  potential  munitions-related  features. 

Potential  features  were  drawn  as  Arcinfo  point  shape  files  for  each  data  set. 

3.7.4  Processing  Lidar  Points  to  Create  Seamless  Surface  Models. 

The  Former  Camp  Beale  lidar  data  offered  an  opportunity  to  test  two  different  data  specifications 
and  their  effects  on  creation  of  surface  models.  For  the  300  m  lidar  flight,  data  was  delivered  in 
non-overlapping  data  blocks,  while  for  the  450  m  lidar  flight,  the  lidar  data  blocks  were 
overlapped  by  10  m.  This  test  is  relevant  because,  while  lidar  vendors  can  produce  overlapping 
point  blocks  without  extra  effort,  there  is  currently  no  industry  standard  as  to  whether  point 
blocks  should  or  should  not  overlap. 
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In  practice,  this  specification  had  a  significant  effect  on  the  creation  of  surface  models.  Non¬ 
overlapping  point  blocks  yielded  “lines”  or  areas  of  no  data  between  the  blocks,  from 
approximately  1  foot  (0.3  m)  in  width  to  2  or  3  meters  at  the  comers.  These  gaps  result  from  the 
operation  of  the  sequential  processing  steps  used  to  create  surface  models,  each  of  which 
increased  the  width  of  the  no-data  area.  The  process  is  shown  in  the  following  images. 


Figure  3-9 

Surface  Model  Gaps  between  Lidar  Blocks 


Lidar  points,  non-overlapping  point 
blocks. 


Step  2:  Digitial  Elevation  Models 
(DEMs)  created  from  TINs.  Gaps  are 
slightly  larger. 


Step  1:  TINs  created  from  non¬ 
overlapping  point  blocks.  Gaps  begin  to 
appear. 
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Step  3:  Hillshade  created  from  DEMs 
(DEMs  showing  underneath).  Gaps  are 
once  again  slightly  larger. 
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In  theory,  this  effect  may  be  overcome  by  combining  all  of  the  lidar  points  into  a  single  file  and 
processing  to  a  single  TIN  and  DEM.  In  practice  such  an  approach  would  create  a  point  file  that 
was  too  large  to  be  processed  using  any  available  software.  A  more  practical  alternative  is  to 
specify  delivery  of  overlapping  point  blocks.  DEMs  are  then  created  from  each  separate  point 
block  and  then  merged.  The  following  images  show  the  result  for  the  same  area  shown  in  the 
previous  images.  The  overlapping  point  files  created  a  surface  model  free  of  gaps. 

Figure  3-10 


Lidar  points  showing  overlapping  point  blocks  Hillshades  from  same  area  showing  no  gaps. 

3.7.5  Detection  and  Delineation  of  Potential  Munitions  Response  Sites  and  Ground 
Features 

Non-Munitions-Related  Features 

Once  the  lidar  data  had  been  converted  to  useable  GIS  products,  the  lidar  and  orthophoto  data 
were  examined  and  individual  features  noted.  The  demonstration  site  contains  numerous  ground 
features  with  a  much  wider  variety  than  were  observed  at  the  Kirtland  PER  or  Victorville  DBT 
“Y”  sites.  Some  site  features  were  clearly  not  related  to  munitions  use.  These  features  could  be 
eliminated  from  further  investigation  with  relatively  high  confidence,  and  were  not  included  as 
features  of  interest.  Some  example  non-munitions-relate  features  are  shown  in  the  following 
images. 
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Figure  3-11 


■  -  - 


Ground  points  surface  model  of  potential 
features. 


All  points  surface  model  showing  house  and 
outbuildings 


Figure  3-12 


Ground  points  surface  model  of  potential  feature.  Orthophoto  showing  water  impoundment 


Features  that  were  potentially  related  to  munitions  use  were  marked  and  assembled  to  a  GIS 
point  file.  A  total  of  989  ground  features  were  identified.  This  did  not  include  the  individual 
potential  craters  in  one  of  the  two  crater  fields,  which  were  too  faint,  numerous,  and  overlapping 
to  be  counted  accurately. 
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Figure  3-13 


First  round  of  feature  identification  assessment. 


The  identified  features  were  then  examined  along  with  data  from  the  CSM,  and  grouped  into 
initial  areas  of  interest.  These  areas  are  diseussed  further  in  Seetion  4.2,  Performanee 
Confirmation  Methods,  in  the  light  of  field  verifieation  aetivities. 


Figure  3-14 


Initial  areas  of  interest. 

Likely  MEC  features:  red 
High  potential  MEC  features:  blue 
Potential  MEC  features:  yellow,  and 
Other  features  of  interest:  green 


FINAL  REPORT  ADDENDUM 
High  Density  Lidar  and  Orthophotography 
in  UXO  Wide  Area  Assessment 
Contract  No.  N44255-02-D-2008 


Section  3.0 
Revision  No.:  2 
Date:  January  2008 
Page  3-15 


3.7.6  Ambiguous  Features 

The  majority  of  ground  features  on  the  demonstration  site  were  ambiguous,  and  their  origins 
could  not  be  determined  from  the  lidar  and  orthophoto  data  alone  with  high  confidence.  Some 
example  ambiguous  features  are  shown  in  the  following  images.  Generally,  these  took  the  form 
of  depressions  from  2  -  5  m  in  size,  appearing  semi-randomly  or  in  a  variety  of  groups. 


Figure  3-15 


Ambiguous  ground  features 


3.7.7  Potential  Munitions  Use  Sites 


Five  areas  on  the  site  could  be  labeled  as  potential  MRS  with  relative  confidence.  These 
included  two  possible  firing  ranges  and  three  potential  bombing  areas. 

Potential  Firing  Range  1 

Potential  Firing  Range  1  is  located  in  the  central  western  portion  of  the  site.  The  site  consists  of 
a  relatively  flat,  square  area  to  the  north,  approximately  39  potential  craters  around  3  m  in 
diameter  and  0.5  m  deep,  and  a  ship-shaped  object  approximately  20  by  40  m.  To  the  south  is  a 
low  hill.  The  area  is  within  Range  8  from  1956  and  Range  7  from  1959,  both  of  which  were 
designated  as  ground  ranges  where  57mm  Recoilless  Rifle,  60mm  Mortar,  .50  caliber  weapons 
and  shape  charges  were  used. 
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Figure  3-16 


3.7.8  Potential  Firing  Range  2 


Potential  Firing  Range  2  is  loeated  in  the  south  part  of  the  study  area.  The  area  eonsists  of  13 
pairs  of  fan-shaped  level  areas,  along  with  7  bunkers.  The  area  is  located  within  the  Primary 
Navy  Toss  Bomb  Target  (1956).  There  is  no  information  available  as  to  the  type  of  munitions 
used  in  the  Navy  Toss  Bomb  Target.  The  area  is  also  within  the  USAF  LG.  Training  Area 
(1956)  and  borders  Targets  2  (1956)  and  4  (1955).  Target  2  was  used  for  high  explosive  charges 
up  to  250  lbs.  The  USAF  LG  training  area  was  used  for  small  arms,  signals,  booby  traps,  trip 
flares  and  other  pyrotechnics. 
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Figure  3-17 


3.7.9  Potential  Bombing  Area  1 


Potential  Bombing  Area  1  is  a  bull’s  eye  target  located  in  the  northern  portion  of  the  site,  and  is 
the  clearest  target  object  on  the  site.  The  target  consists  of  4  rings,  the  largest  of  which  is 
approximately  300  m  (984  ft)  in  diameter.  The  area  contains  numerous  craters  ranging  up  to 
approximately  3  m  in  diameter  and  0.1  m  deep.  This  target  is  located  between  Target  3  and 
Navy  Target  T-63  in  the  CSM,  which  was  used  for  HE  bombing  practice  from  1948  to  1955. 
The  location  of  Bull’s-eye  Target  1  is  not  within  the  mapped  location  of  either  CSM  Target  3  or 
Navy  Target  T-63.  It  appears  most  likely  that  this  is  the  HE  bombing  target,  and  the  maps 
accompanying  the  initial  CSM  are  somewhat  erroneous. 
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Figure  3-19 


Bull’s-eye  Target  1  with  CSM  Target  Areas  3  and  T-63 


3.7.10  Potential  Bombing  Area  2 


Potential  Bombing  Area  2  is  loeated  in  the  north  eentral  portion  of  the  site.  This  area  eonsists  of 
several  hundred  potential  eraters,  approximately  1  m  to  3  m  in  diameter  and  up  to  0.40  m  deep. 
The  target  area  eontains  a  mound  (now  approximately  0.70  m  tall)  that  may  represent  a  target 
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center  point.  The  area  is  within  Target  Area  6  from  the  CSM,  which  was  used  for  live  bomb 
releases  for  Shoran  training  between  1955  and  1959. 


Figure  3-20 


3.7.11  Potential  Bombing  Area  3 


Potential  Bombing  Area  3  is  located  in  the  north  central  portion  of  the  site.  This  area  consists  of 
several  hundred  potential  craters,  approximately  1  m  to  3  m  in  diameter  and  up  to  0.40  m  deep. 
As  with  Potential  Bombing  Area  2,  the  area  is  within  Target  Area  3  from  the  CSM,  which  was 
used  for  live  bomb  releases  for  Shoran  training  between  1955  and  1959.  The  area  may  represent 
a  second  target  area  within  Target  Area  3. 
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3.7.12  Data  Density  Effects  -  Orthophotos 

The  Former  Camp  Beale  demonstration  was  not  designed  to  test  the  effects  of  orthophoto  data 
density.  During  the  first  phase  of  the  WAA  Pilot  Program,  orthophotos  were  collected  at  10  cm 
and  20  cm  pixel  sizes  at  the  Kirtland  PBR  site;  and  based  on  the  results  of  this  test;  collection  at 
subsequent  sites  was  only  at  10  cm.  However,  by  coincidence  the  Army  Corps  of  Engineers 
already  had  available  orthophotography  of  the  site  (in  the  MrSID  compressed  format)  at  a  30  cm 
(1  foot)  pixel  resolution.  Consequently,  it  was  possible  to  compare  this  orthophoto  data  to  the  10 
cm  pixel  data  collected  as  part  of  the  demonstration. 

Figure  3-22 


Bull’s  eye  aiming  target:  10  cm  pixel  orthophoto  Bull’s  eye  aiming  target:  30  cm  (1  ft)  pixel 

orthophoto 
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In  at  least  one  area  of  the  site,  the  difference  between  the  two  data  sets  was  striking.  The  target 
rings  in  Potential  Bombing  Area  1  are  clearly  visible  in  the  10  cm  orthophoto,  but  are  virtually 
undetectable  in  the  30  cm  orthophoto. 

3.7.13  Data  Density  Effects  -  Lidar 

Lidar  data  density  is  based  on  the  average  number  of  lidar  points  per  square  meter.  While  these 
average  values  are  useful  as  general  descriptors,  lidar  data  density  actually  varies  considerably 
over  the  ground  surface,  a  complex  phenomenon  that  is  discussed  in  Appendix  C  of  the  final 
report  for  the  Kirtland  PBR  and  Victorville  DBT  “Y”  sites.  Lidar  data  was  obtained  at  the 
following  overall  data  densities: 


Table  3-2 

Achieved  Lidar  Data  Densities 


Flight 

Overall  point  density  (pts/ni2) 

450  m  flight 

13.8 

300  m  flight 

13.7 

2 

The  achieved  data  densities  are  considerably  higher  than  the  performance  criteria  of  3  pts/m  for 
the  450  m  flight  and  5  pts/  m^  for  the  300  m  flight.  These  high  densities  resulted  from  two 
factors.  First,  the  sensor  speed  was  increased  to  75  kHz,  compared  to  the  45  kHz  and  50  kHz 
used  at  the  previous  sites.  This  decision  was  a  result  of  a  request  to  the  vendor  to  obtain  higher 
densities  if  the  equipment  allowed.  The  second  factor  was  that  flight  speed  was  much  slower 
than  planned  due  to  the  high  ambient  air  temperatures.  These  two  factors  combined  to  yield 
much  higher  densities  than  planned.  In  turn,  these  high  lidar  densities  allowed  for  the  creation  of 
exceptionally  detailed  surface  models.  For  example,  in  the  following  images,  individual  small 
tree  trunks  can  be  seen  on  the  ground  surface. 
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Figure  3-23 


10  cm  orthophoto  showing  fallen  trees  and  rocks 
on  the  ground. 


All  points  lidar  surface  model  hillshade  of  the 
same  area  showing  the  same  objects  in  the  lidar 
image. 


3.7.14  Detection  of  Test  Craters 

At  the  Kirtland  and  Victorville  test  sites,  detection  of  test  craters  using  lidar  improved  as  the  data 
density  increased.  At  both  sites,  test  craters  at  1.0  and  1.5  m  in  diameter  were  reliably  detected 
at  all  data  densities  (1.5-5  pts/m^);  while  the  0.30  m  (1  foot)  test  craters  were  not  detected  at 
any  data  density  acquired. 

At  the  Former  Camp  Beale  site,  results  were  similar  despite  the  higher  data  densities:  the  1.0  m 
and  1.5  m  craters  were  reliably  detected,  while  the  0.30  m  test  craters  were  not,  as  shown  in  the 
following  images. 
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Figure  3-24 

0.3  m 

0.3  m 

0.3  in 

1.5  m  ^ 

0.3  m 

0.3  m 

0.3  m 

1.S  m  ' 

1.0  m  » 

-  -  1.0  m 


All  points  surface  model  showing  test 
craters. 


Ground  returns  surface  model  showing 
test  craters. 


300  m  lidar  points,  (both  ground  and  non-  Portion  of  the  image  to  the  left,  300  m  lidar 

ground).  Several  of  the  0.3  m  craters  appear  points,  (both  ground  and  non-ground).  One 

to  lie  between  the  rows  of  lidar  points.  0.3  m  crater  has  a  lidar  point,  one  does  not. 


These  images  show  that  even  with  a  high  point  density  of  13.8  pts/  m^,  lines  of  individual  lidar 
points  are  frequently  spaeed  suffieiently  far  apart  so  as  to  miss  the  0.3  m  eraters.  While  some 
eraters  of  this  size  ean  be  (and  were)  deteeted,  the  probability  of  deteetion  is  far  from  100% 
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3.7.15  Data  Artifacts  and  Noise  Effects 

As  at  the  previous  demonstration  sites,  the  Former  Camp  Beale  lidar  data  showed  “corduroy” 
effects  roughly  0.05  m  deep  in  areas  of  relatively  flat,  smooth  terrain.  The  size  of  the  anomaly  is 
well  within  the  vertical  accuracy  specifications  for  lidar  data.  This  is  a  common  lidar  artifact, 
generally  believed  to  result  from  small  errors  in  the  GPS,  IMU  and  laser  range  finder  that  cannot 
be  adjusted  out  during  data  processing.  The  images  below  show  the  relationship  between  the 
observed  “corduroy  stripes”  in  the  modeled  ground  surface  and  the  lines  of  lidar  points. 


Figure  3-25 


“Corduroy”  effect  “Corduroy”  effect  with  lidar  points 


A  previously  unnoted  lidar  data  effect  was  encountered  in  the  Former  Camp  Beale  lidar  data. 
Trees  and  other  tall  objects  caused  a  “shadow”  effect  in  the  lidar  points  as  shown  in  the 
following  images. 


Figure  3-26 


Orthophoto  of  tree  within  the  study  Lidar  points  of  the  same  tree.  Lidar  points  of  the  same  tree, 

area  The  flight  line  centerline  is  to  the  The  flight  line  centerline  is  to  the 

left  (west)  of  the  tree.  right  (east)  of  the  tree. 
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This  effect  results  from  the  fact  that  all  but  a  few  lidar  points  approach  the  ground  surface  at  an 
angle  rather  than  straight  down  from  the  aircraft.  Consequently,  lidar  points  can  be  blocked  from 
the  ground  surface  by  trees  or  buildings,  creating  ‘‘shadowed”  areas  with  few  or  no  returns. 
Ground  features  in  such  areas  will  not  be  detected.  This  effect  may  be  somewhat  mitigated  by 
overlapping  flight  lines  or  conducting  two  lidar  flights,  as  shown  in  the  following  images: 
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Figure  3-27 


Orthophoto. 


Lidar  points,  450  m  flight,  showing  different 
areas  with  few  or  no  lidar  points. 


Lidar  points,  300  m  flight,  showing  small 
shadowed  areas  with  few  to  no  lidar  points. 


^ 

Mrfc  t-a 


Lidar  points,  both  flights  combined;  areas  with 
few  or  no  lidar  points  are  largely  eliminated. 


3.7.16  Effects  of  Flight  Line  Orientation 


At  the  Kirtland  site,  faint  roads  were  detected  differently  by  lidar  with  different  flight  line 
orientations.  For  the  Camp  Beale  site,  this  effect  was  not  observed,  possibly  because  all  of  the 
north/south  and  east/west  roads  were  sufficiently  large  and  distinct,  and  possibly  because  of  the 
higher  lidar  data  density  obtained. 
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3.7.17  Vegetation  Effects 

As  a  light-based  technology,  lidar  does  not  penetrate  vegetation.  Nevertheless,  lidar  is  often 
successfully  used  to  model  the  ground  surface  under  vegetation,  since  lidar  points  will  often  fall 
in  the  many  gaps  between  foliage.  In  practice,  lidar  has  been  used  to  model  ground  surfaces  in 
all  but  truly  closed  canopies. 

However,  while  some  lidar  points  will  penetrate  to  the  ground  surface  in  vegetated  areas,  most 
will  not.  Consequently,  the  surface  model  under  vegetative  cover  will  be  less  detailed.  As  a 
preliminary  attempt  to  quantify  this  effect,  two  areas  of  %  km^  were  modeled.  A  grid  of  2  m 
cells  was  created,  with  each  cell  assigned  the  percentage  of  lidar  returns  that  were  reflected  from 
a  point  3  feet  or  higher  from  the  ground  surface.  At  these  two  areas,  the  forested  areas  blocked 
from  50%  -  80%  of  the  lidar  points,  while  in  dense  brush  over  90%  of  the  lidar  points  were 
blocked.  Under  the  trees,  good  modeling  of  the  ground  surface  was  nevertheless  achieved.  This 
may  have  been  a  result  of  the  high  overall  lidar  data  densities  involved.  In  the  area  of  dense 
brush,  the  vegetation  effect  caused  serious  degradation  of  the  surface  model.  The  effect  is  shown 
in  the  following  images. 


Figure  3-28 


Orthophoto  showing  open  area  and  trees.  All  points  lidar  surface  model. 
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Figure  3-29 


Lidar  ground  surface  model.  Density  map  showing  the  percentage  of  lidar  points 

reflected  from  vegetation  over  3  ft.  in  height. 

The  following  images  show  an  area  eontaining  both  trees  and  an  area  of  dense  brush. 


Figure  3-30 


Orthophoto  showing  open  area,  trees  and  brush. 


All  points  lidar  surface  model. 
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Figure  3-31 


Lidar  ground  surface  model,  showing  good  quality  Density  map  showing  the  percentage  of  lidar  points 

for  the  treed  area  and  serious  degradation  for  the  reflected  from  vegetation  over  3  ft  tall, 

brushy  area. 

These  preliminary  results  indicate  that  in  vegetated  areas,  lidar  data  densities  should  be  higher 
and  that  in  areas  with  dense  vegetation,  confidence  in  the  lidar  surface  model  will  be  lower. 
Vegetation  effects  will  be  the  subject  of  further  investigation. 


3.7.18  Lidar  and  Orthophoto  Positional  Accuracy 

Positional  accuracy  specifications  were  established  in  the  Demonstration  Plan  for  each  site,  and 
repeated  in  Table  4-1,  Performance  Criteria.  Lidar  and  orthophoto  data  met  the  positional 
accuracy  criteria  established.  The  following  table  presents  the  overall  positional  accuracy  results 
for  the  two  sites. 


Item 

Performance  Criteria 

Results  1 

[m) 

Lidar  vertical  accuracy 

Avg.  dz:  +/-  0.15  m  compared  to 
control  points 

Lidar  to  ESTOP  control  points 

300  m  flight 

450  m  flight 

Avg.  dz:  0.008  m 

Avg.  dz:  0.095  m 

Lidar  horizontal  accuracy 

Avg.  dx/y:  +/-  0.65  m  compared 
to  control  points 

Average  x  and  y  displacement  (dx 
and  dy)  for  all  control  points  for 
each  flight. 

300/450  m  flights 

Avg.  dx:  0.15  m 

Avg.  dy:  0.21  m 

Orthophoto  horizontal 
accuracy 

Avg.  dx/y  under  3  pixel  widths 
compared  to  control  points 

10  cm  orthophotos  to  control  points 

450  m  flight 

Avg.  dx:  0.19  m 

Avg.  dy:  0.24  m 

Orthophoto  to  lidar  alignment 

Avg.  dx/dz  under  2  pixel  widths 

10  cm  orthophotos  to  lidar  positions 

Avg.  dx:  0.13  m 

Avg.  dy:  0.10  m 
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4.0  PERFORMANCE  ASSESSMENT 

4.1  PERFORMANCE  CRITERIA 

Lidar  and  orthophoto  data  collected  for  this  demonstration  met  the  performance  criteria  related  to 
data  collection,  data  procession,  site  coverage  and  positional  accuracy. 

MRS  were  detected  and  the  location  of  one  bombing  target  was  corrected  from  the  CSM. 
Numerous  ground  features  were  identified.  The  origin  of  many  features  could  not  be  determined 
from  the  lidar  and  orthophoto  data  alone;  however,  lidar  and  orthophoto  data  was  usable  to 
identify  features  for  further  investigation. 

The  following  performance  objectives  are  taken  from  the  Technology  Demonstration  Plan  for  the 
site. 


Table  4-1 

Performance  Criteria 


Performance  Criteria 

Description 

Primary  or 
Secondary 

Pre-mobilization 

Verification  of  survey  control 
point  positions 

Verify  survey  control  point  locations  within  at  least  3rd  order 
accuracy. 

Primary 

Lidar  data  collection  and  processing 

Area  coverage 

100%  coverage  for  each  flight. 

Primary 

Lidar  point  density 

Achieve  overall  lidar  point  densities  of: 

300  m  flight  (1)  -  5  pts/  m^ 

450  m  flight  (1)  -  3  pts/  m^ 

Primary 

Lidar  vertical  accuracy 

Vertical  accuracy  of  +/-  15  cm  compared  to  ground  survey. 

Primary 

Lidar  horizontal  accuracy 

Horizontal  accuracy  of  +/-  65  cm  compared  to  ground  survey. 

Primary 

Orthophoto  data  collection  and  processing 

Orthophoto  area  coverage 

100%  coverage  for  each  flight. 

Primary 

Orthophoto  flight  altitude  / 
pixel  size 

450  m  (for  10  cm  pixel  flight). 

Primary 

Orthophoto  horizontal 
alignment  to  Lidar 

Lidar  and  orthophotos  aligned  so  that  target  features  are  not 
displaced  in  the  two  data  sets. 

Primary 

Orthophoto  horizontal 
alignment  to  survey  control 
points 

Orthophotos  aligned  to  survey  control  points  so  that  target 
features  are  not  displaced. 

Primary 

Munitions  Response  Site  identification  and  analysis 

MRS  identification 

Correctly  identify  MRS  identified  in  the  CSM. 

Primary 

MRS  false  alarm  rate 

No  areas  incorrectly  identified  as  MRS. 

Primary 

MRS  boundary  delineation 

Correctly  locate  MRS  boundaries  to  +/-  15%  of  ground-truthed 
area. 

Primary 

MRS  feature  identification 

Identify  features  presenting  as  munitions  related 
(anthropogenic) 

Primary 

FINAL  REPORT  ADDENDUM  Section  4.0 

High  Density  Lidar  and  Orthophotography  Revision  No.:  2 

in  UXO  Wide  Area  Assessment  Date:  January  2008 

Contract  No.  N44255-02-D-2008  Page  4-2 

4.2  PERFORMANCE  CONFIRMATION  METHODS 

4.2.1  Demonstration-Level  Confirmation  Methods 

At  the  demonstration  level,  effectiveness  of  lidar  and  orthophotos  was  evaluated  based  on  its 
ability  to  meet  the  stated  performance  criteria  given  in  Table  4-1.  The  demonstration  relied  on 
proven  industry  methods  to  assure  predictable  results,  including  the  use  of  survey  controls, 
equipment  calibration,  alignment  of  lidar  points  to  the  survey  control  points  and  from  one  flight 
line  to  the  next,  and  QA/QC  checks  throughout  the  project.  Both  lidar  and  orthophoto  data  met 
all  data  quality  specifications. 

4.2.2  Program  Level  Confirmation  Methods 

In  order  to  investigate  some  of  the  ambiguous  features,  a  field  crew  visited  the  site  between 
February  2“'^  and  the  5**^,  2007.  Over  three  days  of  field  work,  134  ground  features  identified 
from  the  lidar  and  orthophoto  data  were  visited,  photographed  and  examined  using  a  Schondstedt 
hand-held  magnetometer.  The  field  crew  included  senior-level  UXO  staff  with  many  years  of 
field  experience  on  a  variety  of  sites.  Field  work  was  limited  to  areas  where  site  access  could  be 
obtained,  which  excluded  most  potential  areas  of  interest. 


4.2.3  Distinguishing  MEC  vs.  Non-MEC  Features 

Once  in  the  field,  some  ambiguous  features  could  be  resolved  with  a  relatively  high  level  of 
confidence.  Some  of  the  characteristics  of  the  ground  features  could  be  applied  to  subsequent  re¬ 
analysis  of  the  lidar  and  orthophoto  data  in  the  office,  including  the  following: 

Shape.  Some  depressions  were  more  rectangular  than  round,  and  sharply  resembled  depressions 
from  earth-moving  machinery.  These  depressions  did  not  resemble  craters.  This  rectangular 
shape  could  often  be  seen  in  the  lidar  data,  especially  at  the  high  lidar  data  density  for  this  site. 
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Figure  4-1 


Rectangular  ground  feature.  Shape  did  not  resemble  crater. 


Figure  4-2 


Rectangular  ground  feature.  Shape  did  not  resemble  crater. 


Size.  Some  features  that  were  roughly  eireular  were  too  large  to  be  likely  eraters.  This  was 
espeeially  true  where  sueh  eraters  appeared  in  areas  with  no  history  of  HE  use. 
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Figure  4-3 


Roughly  circular  depression,  roughly  10  m  in  diameter.  Likely  too  large  to  be  likely  a  crater. 


Clustering.  Individual  depressions,  loeated  away  from  other  depressions  or  other  features,  were 
judged  to  less  likely  to  be  related  to  munitions  use. 


Figure  4-4 
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Clustered  potential  craters. 
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Isolated  depression  away  from  other  features. 


Correlation  with  CSM  target  areas.  The  Former  Camp  Beale  site  contains  numerous, 
overlapping  ranges,  including  bombing  ranges,  firing  ranges  and  training  ranges.  Information 
regarding  the  historic  use  of  many  of  these  ranges  is  scanty,  and  sometimes  contradictory.  For 
the  purposes  of  validating  feature  classifications,  the  procedure  used  was  as  follows: 

•  When  looking  at  potential  craters,  only  HE  bombing  ranges  were  considered.  These 
included  Target  2  (250  lb).  Target  3  (100  lb).  Navy  T-63  (25,  possible  250  lb  and  500  lb) 

•  When  looking  at  the  one  bull’s  eye  ring,  only  Target  3  was  considered.  This  target  was 
used  for  100  lb  HE  bombs  and  practice  bombs,  and  the  CSM  states  that  remnants  of  a 
bull’s  eye  ring  were  detected  in  the  NE  portion  of  the  site.  This  is  consistent  with  the 
location  of  Potential  Bombing  Area  1,  NE  of  and  outside  the  boundary  of  Target  3. 

•  When  looking  at  other  features  other  than  potential  craters,  firing  ranges  and  training 
ranges  were  considered  if  their  use  appeared  consistent  with  the  types  of  features 
observed. 

Other  characteristics  were  clear  in  the  field  but  more  difficult  to  apply  to  subsequent  analysis  of 
the  lidar  and  orthophoto  data.  In  particular,  some  small,  round  depressions  were  sometimes 
found  to  be  old  mine  shafts.  In  the  lidar  and  orthophoto  data,  it  remained  difficult  to  identify 
these  correctly. 
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Figure  4-5 


Suspected  mine  shaft. 


Some  features  showed  positive  magnetometry  responses,  and  one  piece  of  ordnance-related  scrap 
was  found  on  the  surface  during  the  field  visit.  However,  some  crater-shaped  features  had 
positive  and  others  negative  magnetic  response,  and  there  was  no  clear  correlation  between 
magnetic  response  and  the  shape  of  the  feature.  This  finding  reinforces  the  necessity  of 
following  the  use  of  lidar  and  orthophotos  with  technologies  such  as  magnetometry  or  EMI  that 
directly  detect  ordnance  components. 


Figure  4-6 


Ground  feature  with  negative  magnetic  response. 
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Figure  4-7 


Ground  feature  with  positive  magnetic  response. 


Of  the  134  features  examined  in  the  field,  118  (88%%)  showed  no  evidence  of  munitions  use, 
and  16  (12%)  showed  positive  magnetic  response  or  other  evidence  of  munitions  use.  These 
results  were  judged  to  have  relatively  high  confidence. 

Following  the  field  visit,  the  UXO  staff  person  who  had  led  the  field  work  examined  the  lidar 
and  orthophoto  data  for  the  features  that  had  not  been  visited,  and  applied  the  results  of  the  field 
visit,  including  the  factors  discussed  above,  to  make  a  professional  judgment  as  to  the  likelihood 
of  munitions  origin.  Of  the  856  features  that  were  not  visited,  583  (68%)  were  evaluated  as  no 
evidence  of  munitions  use  and  273  (32%)  were  evaluated  as  probable  munitions  use. 

These  office  evaluations  were  judged  to  have  only  medium  confidence,  based  on  the  lack  of 
magnetometry  results  and  the  finding  that  potential  craters  with  and  without  positive  magnetic 
response  could  have  similar  shapes.  Further  site  investigation  would  be  warranted  to  increase 
this  confidence  level. 
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4.2.4  Correlation  with  Areas  of  Interest 


Once  the  individual  features  had  been  re-evaluated,  the  original  Areas  of  Interest  were  also  re¬ 
evaluated  in  the  light  of  the  field  visit,  and  their  assessments  updated. 

Figure  4-8 


Results  from  Camp  Beale  Site  Visit 


Examination  of  the  original  areas  of  interest  in  the  light  of  the  field  results  showed  that  the 
original  assignments  were  only  roughly  accurate.  This  finding  is  consistent  with  the  ambiguous 
nature  of  many  of  the  features.  The  following  images  illustrate  some  of  the  comparisons 
between  the  field  work  and  the  original  areas  of  interest. 
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Figure  4-9  -  Initial  and  Subsequent  Analysis 


Initial  Areas  of  Interest 


Kilometers 


Red:  Likely  MEC  Features 

Blue:  High  Potential  MEC  Features 

Vellow:  Potential  MEC  Features 
Green :  Other  F eatures  of  Interest 
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In  the  figures  that  follow,  the  following  symbols  show  whether  the  feature  was  inspected  during 
the  site  visit,  and  whether  either  the  field  visit  or  subsequent  inspection  by  UXO  staff  showed 
evidence  of  munitions  use: 

Not  Visited,  No  Evidence  of  Munitions  Use 
N  Not  Visited,  Probable  Munitions  Use 

Visited,  No  Evidence  of  Munitions  Use 
E  Visited,  Probable  Munitions  Use 
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Figure  4-10.1  -  Area  of  Interest  1 


Area  of  Interest:  1 

Original  Classification:  Potential  MEC  Features. 
Original  classification  based  on:  large  numbers  of  crater¬ 
shaped  depressions. 

CSM  targets:  None 

Field  Visit:  Not  visited;  no  site  access. 

Revised  classification:  Other  Features  of  Interest 
Revised  classification  based  on: 

No  CSM  targets 

Depressions  are  larger  than  typical  ordnance  craters 
Depressions  are  relatively  scattered 
Confidence  level:  Medium,  no  field  visit 
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Figure  4-10.2  -  Area  of  Interest  2 


Area  of  Interest:  2 

Original  Classification:  Other  Features  of  Interest 
Original  Classification  based  on:  depressions  evident  but 
scattered  and  with  no  apparent  patterns 
CSM  targets:  Night  Bombing  Target  No.  2  (1956/9) 

Field  Visit:  Not  visited;  no  site  access 
Revised  classification:  Other  Features  of  Interest 
Revised  classification  based  on:  No  documented  use  of  most 
CSM  targets.  Depressions  are  larger  than  typical  ordnance 
craters.  Depressions  are  often  not  circular.  Depressions  are 
relatively  scattered. 

Confidence  level:  Medium,  no  field  visit 
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Figure  4-10.3  -  Area  of  Interest  3 


Area  of  Interest:  3 

Original  Classification:  High  Potential  of  MEC  Features 
Original  Classification  based  on:  large  numbers  of  crater-like 
depressions  in  close  proximity  to  obvious  targets. 

CSM  targets:  Range  2  (1955),  Bombing  Targets  Nos.  2  &  3 
(1955/9),  Night  Bombing  Target  No.  2  (1956),  Target  No.  6 
(1956/9),  Navy  Bombing  Target  T-63  (1956) 

Field  Visit:  Not  visited;  no  site  access 
Revised  classification:  High  Potential  of  MEC  Features 
Revised  classification  based  on:  Documented  use  of  most 
CSM  targets  .  Many  depressions  have  size  and  shape  consistent 
with  ordnance  craters,  others  do  not. 

Confidence  level:  Medium,  no  field  visit.  In  particular, 
assessment  of  some  individual  depressions  is  sufficiently 
ambiguous  to  warrant  further  validation. 
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Figure  4-10.4  -  Area  of  Interest  4 


^ Meters 


Area  of  Interest:  4.  This  is  Potential  Bombing  Area  1. 

Original  Classification:  Likely  MEC  Features 

Original  Classification  based  on:  Numerous  crater-like 

depressions  within  apparent  target  rings  centered  on  small  mound 

in  the  center  of  the  rings 

CSM  targets:  Range  2  (1955),  Range  8  (1956) 

Field  Visit:  Not  visited;  no  site  access 
Revised  classification:  Likely  MEC  Features 
Revised  classification  based  on:  Depressions  have  sizes  and 
shapes  consistent  with  ordnance  use.  Depressions  are  associated 
with  target  rings.  Although  the  target  rings  are  outside  of  the 
mapped  areas  for  ranges  with  HE  use  the  feature  is  clearly  a 
bombing  target 
Confidence  level:  High 
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Figure  4-10.5  -  Area  of  Interest  5 


Meters 


Area  of  Interest:  5.  This  is  Potential  Bombing  Area  2 
Original  Classification:  Likely  MEC  Features 
Original  Classification  based  on:  large  numbers  of  crater¬ 
shaped  depressions  relatively  clustered. 

CSM  targets:  Range  2  (1955),  Bombing  Target  3  (1955),  Range 
8  (1956),  Target  No.  6  (1956/9) 

Field  Visit:  Not  visited;  no  site  access 
Revised  classification:  Likely  MEC  Features 
Revised  classification  based  on:  Depressions  have  sizes  and 
shapes  consistent  with  ordnance  use.  Depressions  are  highly 
clustered  in  a  pattern  consistent  with  bombing  practice. 
Depressions  are  located  in  CSM  Bombing  Target  3  which  had 
documented  HE  use. 

Confidence  level:  High 
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Figure  4-10.6  -  Area  of  Interest  6 


Area  of  Interest:  6.  This  is  Potential  Bombing  Area  3 
Original  Classification:  Likely  MEC  Features 
Original  Classification  based  on:  large  numbers  of  crater¬ 
shaped  depressions  relatively  clustered. 

CSM  targets:  Range  2  (1955),  Bombing  Target  3  (1955),  Target 
No.  6  (1956/9) 

Field  Visit:  Not  visited;  no  site  access 
Revised  classification:  Likely  MEC  Features 
Revised  classification  based  on:  Depressions  have  sizes  and 
shapes  consistent  with  ordnance  use.  Depressions  are  highly 
clustered  in  a  pattern  consistent  with  bombing  practice. 
Depressions  are  located  in  CSM  Bombing  Target  3  which  had 
documented  HE  use. 

Confidence  level:  High 
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Figure  4-10.7  -  Area  of  Interest  7 
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Meters 

Area  of  Interest:  7 

Original  Classification:  Likely  MEC  Features 

Original  Classification  based  on:  large  number  of  crated- 

shaped  depressions,  relatively  clustered 

CSM  targets:  Bombing  Targets  Nos.  2  &  3  (1955),  Night 

Bombing  Target  No.  2  (1956),  Bombing  Target  No.  6  (1956) 

Field  Visit:  Not  visited;  no  site  access 

Revised  classification:  Potential  MEC  Features 

Revised  classification  based  on:  Depressions  are  not  always 

circular,  and  are  not  in  a  pattern  typical  of  bombing  practice  or 

OB/OD  use,  nor  are  they  near  obvious  roads.  Depressions  are 

close  to  AOI  6,  a  potential  bombing  target. 

Confidence  level:  Medium,  no  field  visit.  This  area  is  still 
ambiguous  and  warrants  additional  validation. 
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Figure  4-10.8  -  Area  of  Interest  8 


Meteis 

Area  of  Interest:  8 

Original  Classification:  Potential  MEC  Features 

Original  Classification  based  on:  Two  closely  spaced  groups  of 

depressions 

CSM  targets:  Primary  Toss  Bomb  target  (1956)  (No  evidence  of 
actual  use) 

Field  Visit:  Not  visited;  no  site  access 

Revised  classification:  Other  Features  of  Interest 

Revised  classification  based  on:  Depressions  are  not  clustered 

in  a  manner  consistent  with  bombing  practice  or  OB/OD  use,  nor 

are  they  near  obvious  roads.  No  correlation  with  CSM  targets. 

Confidence  level:  Medium,  no  field  visit.  This  area  is  still 

ambiguous  and  warrants  additional  validation. 
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Figure  4-10.9  -  Area  of  Interest  9 


200 


Area  of  Interest:  9 

Original  Classification:  Potential  MEC  Features 
Original  Classification  based  on:  Ground  surface  feature 
representing  a  potential  firing  point 

CSM  targets:  Primary  Toss  Bomb  target  (1956)  (No  evidence  of 
actual  use) 

Field  Visit:  Not  visited;  no  site  access 
Revised  classification:  Other  Features  of  Interest 
Revised  classification  based  on: 

The  feature  does  not  resemble  a  gun  emplacement  and  appears  to 
be  more  consistent  with  sand  and  gravel  mining 
Confidence  level:  Medium,  no  field  visit 
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Figure  4-10.10  -  Area  of  Interest  10 
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Meters 

Area  of  Interest:  10 

Original  Classification:  Likely  MEC  Features 
Original  Classification  based  on:  Regularly  space  groups  of 
potential  craters 

CSM  targets:  No  HE  bombing  targets 
Field  Visit:  Not  visited;  no  site  access 
Revised  classification:  Other  Features  of  Interest 
Revised  classification  based  on:  Depressions  are  clustered  in  a 
manner  that  is  not  consistent  with  ordnance  use  and  appear  more 
consistent  with  mining  exploration.  Depressions  are  not  near 
obvious  roads  as  would  be  expected  with  OB/OD  use 
Confidence  level:  Medium,  no  field  visit.  This  area  is  still 
ambiguous  and  warrants  additional  validation. 
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Figure  4-10.11  -  Area  of  Interest  11 


Area  of  Interest:  1 1 

Original  Classification:  Potential  MEC  Features 
Original  Classification  based  on:  Presence  or  crater-like 
depressions  following  a  mostly  linear  pattern 
CSM  targets:  Ranges  2  &  4  (1955) 

Field  Visit:  Not  visited;  no  site  access 
Revised  classification:  Likely  MEC  Features 
Revised  classification  based  on:  Many  of  the  depressions  have 
shapes  and  sizes  consistent  with  ordnance  use.. 

However,  CSM  targets  did  not  include  HE  use. 

Confidence  level:  Medium,  no  field  visit.  This  area  is  still 
ambiguous  and  warrants  additional  validation. 
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Figure  4-10.12  -  Area  of  Interest  12 


Original  Classification:  Potential  MEC  Features 

Original  Classification  based  on:  Presence  of  a  number  of 

crater-like  depressions 

CSM  targets:  No  HE  bombing  targets 

Field  Visit:  Not  visited;  no  site  access 

YZ  _ _  Revised  classification:  Potential  MEC  Features 

%  Revised  classification  based  on:  Depressions  are  widely 

^  scattered.  Most,  but  not  all,  depressions  have  sizes  and  shapes 

that  are  not  consistent  with  ordnance  use.  CSM  targets  did  not 
'  include  HE  use. 

Confidence  level:  Medium,  no  field  visit.  This  area  is  still 
ambiguous  and  warrants  additional  validation. 
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Figure  4-10.13  -  Area  of  Interest  13 


200 


Area  of  Interest:  13 

Original  Classification:  Potential  MEC  Features 
Original  Classification  based  on:  Non-natural  surface  feature 
representing  a  potential  firing  point. 

CSM  targets:  Range  5  (1955),  Ranges  8  &  11  (1956),  57mm 
Recoilless  Rifle/60mm  Mortar/. 50  Cal.  Machine  gun  range 
(1959) 

Field  Visit:  Not  visited;  no  site  access 

Revised  classification:  Other  Features  of  Interest 

Revised  classification  based  on:  The  feature  does  not  resemble 

a  gun  emplacement  and  appears  to  be  more  consistent  with 

agriculture-related  water  impoundment. 

Confidence  level:  Medium,  no  field  visit 
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Figure  4-10.14  -  Area  of  Interest  14 
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Figure  4-10.15  -  Area  of  Interest  15 


Meters 


Area  of  Interest:  15.  This  is  Potential  Firing  Range  1 

Original  Classification:  Likely  MEC  features 

Original  Classification  based  on:  Large  number  of  depressions 

clustered  on  and  around  what  appeared  to  be  a  simulated  ship 

target  scored  on  the  ground  surface 

CSM  targets:  Ranges  8  &  1 1  (1956),  57mm  Recoilless 

Rifle/60mm  Mortar/.50  Cal.  Machine  gun  range  (1959) 

Field  Visit:  Site  was  visited 

Revised  classification:  High  Potential  MEC  Features 
Revised  classification  based  on:  Positive  mag  results  during  site 
visit.  Clustering  of  depressions  is  consistent  with  a  firing  range 
Presence  of  apparent  target  object  and  flat  firing  area  to  the  north 

Confidence  level:  High 
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Figure  4-10.16  -  Area  of  Interest  16 
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Area  of  Interest:  16 

Original  Classification:  Other  Features  of  Interest 

Original  Classification  based  on:  Relatively  scarce  and  random 

placement  of  crater-shaped  depressions 

CSM  targets:  No  HE  bombing  targets 

Field  Visit:  Part  of  the  site  was  visited 

Revised  classification:  Potential  MEC  Features 

Revised  classification  based  on:  Positive  mag  response  at  one  of 

the  depressions  visited,  negative  response  at  others.  Depressions 

not  visited  are  clustered  in  a  manner  that  is  not  consistent  with 

bombing  practice  or  OB/OD  use,  nor  are  they  near  obvious  roads. 

Confidence  level:  High  (for  the  portions  of  the  site  visited. 

Medium  (for  the  portions  not  visited).  The  portion  of  the  site  not 

visited  remains  ambiguous  and  warrants  further  validation. 
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Figure  4-10.17  -  Area  of  Interest  17 
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Area  of  Interest:  17 

Original  Classification:  Potential  MEC 

Features 

Original  Classification  based  on:  Non-natural  surface  feature 
representing  a  potential  firing  point. 

CSM  targets:  Range  5  (1955),  Proposed  Toss  Bomb  target 
(1956),  Range  1 1  (1956^  USAF  LG.  Training  Area  (1956), 
Ground  to  Air  Gunnery  Range  13  (1959) 

Field  Visit:  Site  was  not  visited 

Revised  classification:  Potential  MEC  Features 

Revised  classification  based  on:  The  feature  does  not  resemble 

a  gun  emplacement  and  appears,  on  inspection  of  the  ortho 

photo,  to  be  a  small  arms  firing  range. 

Confidence  level:  Medium,  no  field  visit 
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Figure  4-10.18  -  Area  of  Interest  18 


Meters 


Area  of  Interest:  18.  This  is  a  public  pistol  firing  range. 
Original  Classification:  Potential  MEC  Features 
Original  Classification  based  on:  Series  of  berms  thought  to 
part  of  a  training  range 

CSM  targets:  Range  5  (1955),  Range  11  (1956),  USAF  LG. 

Training  Area  (1956),  .30  Cal.  Close  Combat  Range  11  (1959) 

Field  Visit:  Site  was  visited 

Revised  classification:  Likely  MEC  Features 

Revised  classification  based  on:  Information  from  US  ACE  site 

staff  was  that  this  was  a  public  target  range.  Site  visit  confirmed 

this  use. 

Confidence  level:  High 
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Figure  4-10.19  -  Area  of  Interest  19 
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Figure  4-10.20  -  Area  of  Interest  20 


Area  of  Interest:  20.  This  is  Potential  Firing  Range  2. 

Original  Classification:  Likely  MEC  Features 
Original  Classification  based  on:  The  area  contains  13  pairs  of 
fan-shaped  surface  features  that  resembled  potential  firing  points, 
and  7  features  resembling  bunkers 

CSM  targets:  Target  4  (1955),  Target  No.  2  (1956),. 30  Cal. 
Close  Combat  Range  11  (1959) 

Field  Visit:  Site  was  visited 

Revised  classification:  Likely  MEC  Features 

Revised  classification  based  on:  Positive  mag  response  at  some 

depressions  visited.  Site  visit  identified  the  fan-shaped  surface 

features  as  firing  points.  Presence  of  bunkers  was  verified 

Confidence  level:  High 
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Figure  4-10.21  -  Area  of  Interest  21 


Area  of  Interest:  21 

Original  Classification:  Other  Features  of  Interest 
Original  Classification  based  on:  Presence  of  scattered  berms 
and  crater  shaped  depressions  but  no  distinct  patterns 
CSM  targets:  Proposed  Toss  Bomb  (1956)  (No  evidence  of 
actual  use) 

Field  Visit:  Site  was  visited 

Revised  classification:  Other  Features  of  Interest 

Revised  classification  based  on:  No  mag  response  at  any  of  the 

depressions  visited.  Depressions  are  scattered  in  a  manner  not 

consistent  with  ordnance  use.  Some  depressions  are  not  circular. 

Confidence  level:  High 
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Figure  4-10.22  -  Area  of  Interest  22 


Area  of  Interest:  22 

Original  Classification:  Potential  MEC  features 
Original  Classification  based  on:  Potential  shallow  cratering 
under  trees 

CSM  targets:  Target  No.  2  (1956) 

Field  Visit:  Site  was  visited 
Revised  classification:  Other  features  of  interest 
Revised  classification  based  on:  Site  visit  revealed  ground 
disturbance  but  no  obvious  cratering.  Surface  features  may  be 
lidar  artifacts  of  tree  coverage  and  ground  debris. 

Confidence  level:  High 
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Figure  4-10.23  -  Area  of  Interest  23 
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Figure  4-10.24  -  Area  of  Interest  24 
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Figure  4-10.25  -  Area  of  Interest  25 
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Figure  4-10.26  -  Area  of  Interest  26 


Area  of  Interest:  26 

Original  Classification:  Other  features  of  interest 
Original  Classification  based  on:  Linear  surface  feature  with 
scattered  depressions 

CSM  targets:. 30  Cal.  Close  Combat  Range  11  (1959) 

Field  Visit:  Not  visited 

Revised  classification:  Potential  MEC  Features 
Revised  classification  based  on:  Depressions  are  scattered  and 
most  do  not  resemble  ordnance  use.  One  depression  appears 
more  likely  to  be  ordnance-related  based  on  its  size  and  shape. 
Confidence  level:  Medium  -  low,  no  field  visit.  Because  UXO 
staff  identified  one  potential  MEC  feature,  the  area  remains 
ambiguous  and  warrants  further  validation. 
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Figure  4-10.27  -  Area  of  Interest  27 


Area  of  Interest:  27 

Original  Classification:  Other  features  of  interest 

Original  Classification  based  on:  Regular  line  of  depressions  in 

seasonal  stream  course 

CSM  targets:  No  HE  bombing  targets 

Field  Visit:  Not  visited 

Revised  classification:  Other  features  of  interest 
Revised  classification  based  on:  Depressions  are  not  clustered 
in  a  manner  consistent  with  ordnance  use,  nor  are  they  near 
obvious  roads.  There  is  no  CSM  correlation. 

Confidence  level:  Medium,  no  field  visit 


FINAL  REPORT  ADDENDUM 
High  Density  Lidar  and  Orthophotography 
in  UXO  Wide  Area  Assessment 
Contract  No.  N44255-02-D-2008 


Section  4.0 
Revision  No.:  2 
Date:  January  2008 
Page  4-38 


Figure  4-10.28  -  Area  of  Interest  28 


Meters 


Area  of  Interest:  28 

Original  Classification:  Other  features  of  interest 
Original  Classification  based  on:  Scattered  depressions 
CSM  targets:  Target  No.  2  (1956/9) 

Field  Visit:  Part  of  site  was  visited 

Revised  classification:  Potential  MEC  features 

Revised  classification  based  on:  Negative  mag  response  at  all 

features  visited.  Most  depressions  had  sizes  and  shapes  that  were 

not  consistent  with  ordnance  use.  Some  features  not  visited 

appeared  probable  MEC  on  subsequent  review  based  on  size  and 

shape. 

Confidence  level:  High  (for  the  portions  of  the  site  visited, 
Medium  (for  the  portions  not  visited).  The  portion  of  the  site  not 
visited  remains  ambiguous  and  warrants  further  validation. 
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4.3  DATA  ANALYSIS,  INTERPRETATION  AND  EVALUATION 

4.3.1  General  Observations 

The  Former  Camp  Beale  site  was  a  logical  extension  of  the  WAA  Pilot  Program  to  a  more 
complex  site.  The  site  was  more  challenging  in  at  least  two  dimensions.  First,  the  area  had  been 
used  for  a  wider  variety  of  munitions-related  activities  than  the  previous  sites,  including  not  only 
bombing  ranges  but  also  firing  and  training  ranges.  Second,  the  site  was  used  for  a  much  wider 
variety  of  non-munitions-related  activities,  especially  including  mining  exploration.  As  a  result 
of  both  of  these  factors,  the  Former  Camp  Beale  site  presented  a  much  wider  range  of  potential 
features.  Nevertheless,  the  overall  objectives  of  the  demonstration  were  met: 

•  Lidar  and  orthophotos  were  used  to  identify  potential  bombing  targets  and  firing 
ranges  with  a  high  level  of  confidence. 

•  Lidar  and  orthophotos  were  used  to  correct  what  appeared  to  be  an  erroneous  target 
location  in  the  initial  CSM. 

•  Lidar  and  orthophoto  data  were  used  to  produce  lists  and  locations  of  ambiguous 
features  for  further  investigation. 

•  Lidar  and  orthophoto  data  provided  information  on  topography  and  vegetation  that 
was  used  to  plan  magnetometry  and  EMI  transects. 

The  primary  difference  between  the  Former  Camp  Beale  site  and  first  phase  sites  was  that  at 
Former  Camp  Beale  there  were  a  larger  number  of  features  whose  origins  could  not  be 
determined  using  the  lidar  and  orthophoto  data  alone.  The  history  of  mining  exploration  at  the 
site  was  particularly  problematic,  since  this  activity  produced  depressions  that  could  not  easily  be 
distinguished  from  potential  craters.  However,  the  origin  of  most  or  all  of  these  features  could 
be  resolved  with  field  investigation  using  hand-held  magnetometry. 

These  results  emphasize  the  appropriate  use  of  lidar  and  orthophotos  at  the  beginning  of  the  site 
investigation  process,  and  the  importance  of  following  the  use  of  lidar  and  orthophotos  with 
technologies  such  as  magnetometry  and  EMI  that  directly  detect  ordnance  components. 

4.3.2  Correlations  with  Operating  Parameters  and  Performance  Specifications 

For  orthophotos,  the  demonstration  verified  that  10  cm  pixel  orthophotos  performed  better  than 
the  30  cm  pixel  orthophotos  already  available.  This  finding  is  significant  since  30  cm  pixel 
orthophotos  are  the  highest  density  images  that  are  generally  available  to  site  managers. 
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For  lidar,  while  higher  lidar  data  density  produced  very  detailed  ground  surface  models,  they  did 
not  perform  notably  better  than  the  higher  data  densities  collected  at  the  Kirtland  PBR  or 
Victorville  DBT  “Y”  sites  in  terms  of  actual  feature  detection.  Test  craters  at  1.0  m  and  1.5  m 
diameters  were  reliably  detected,  but  even  at  the  high  data  densities  achieved,  0.30  m  (1  ft.) 
diameter  test  craters  could  be  missed. 

Nevertheless,  there  are  some  reasons  to  acquire  additional  lidar  density  if  the  project  budget  and 
the  vendor’s  equipment  allow.  Additional  lidar  density  did  help  to  define  features  more  clearly, 
and  work  at  the  Kirtland  PBR  and  Victorville  DBT  “Y”  demonstration  sites  showed  that  lidar 
data  density  varies  considerably  over  the  project  area,  with  individual  small  areas  well  above  and 
below  the  target  density.  Increasing  overall  lidar  density  is  one  way  to  compensate  for  areas  that 
are  less  dense  than  planned. 

4.3.3  Optimum  Operating  Conditions  and  Appropriate  Uses  of  the  Technologies 

The  results  from  the  Former  Camp  Beale  site  re-enforce  the  general  premise  of  the  WAA  Pilot 
Program:  lidar  and  orthophotos  should  be  the  first  technologies  to  be  deployed  after  completion 
of  the  ASR  and  the  initial  CSM.  At  all  three  sites  examined,  lidar  and  orthophotos  were 
successful  at  revealing  and  verifying  the  broad  picture  of  munitions  use.  Lidar,  especially,  was 
very  successful  at  delineating  targets  and  crater  fields,  as  well  as  additional  areas  of  interest  that 
warranted  investigation.  The  two  technologies  complimented  each  other  well,  each  providing 
data  that  the  other  did  not.  Since  vendors  generally  deploy  these  two  technologies  together,  it 
makes  sense  to  acquire  both  at  future  production  sites. 

The  demonstration  also  supports  the  WAA  Pilot  Program  assumption  that  lidar  and  orthophoto 
acquisition  should  be  followed  with  technologies  that  directly  detect  ordnance  components.  This 
was  especially  true  for  the  Former  Camp  Beale  site,  where  so  many  features  were  ambiguous. 
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Table  4-2 

Data  Quality  Metrics,  Individual  Performance  Measures 


Analytical 

Objective 

Metric 

Action  to 

Achieve  Metric 

Sampling 
Frequency  or 
Timing 

Desired 

Result 

Actual  Result 

Pre-flight  Activities 

1 

Study  area 

boundary 

delineation 

Site  boundary  polygon 
characteristics  agreed 
upon  and  documented 
to  allow  comparison 
with  data  collected. 

Achieve  stakeholder  agreement 
to  boundary  parameters. 

Once,  start  of 
program. 

Document  site 
boundary  for 
measurement  of 
future  boundary 
metrics. 

Accomplished.  Boundaries  were 
agreed  upon  and  utilized. 

2 

Survey  control 
point 

confirmation 

measurement 

Confirm  coordinates  of 
survey  control  points 
within  at  least  3rd 
order  accuracy. 

Perform  and  record  GPS  survey 
(static  or  kinematic). 

Pre-flight  (or 
during  on-site 
acquisition). 

Confirm 
coordinates  of 
survey  control 
points. 

Accomplished.  Control  points  were 
independently  occupied  by  TRSI 
and  ESTCP. 

Lidar  Data  Collection  and  Processing 

3 

Sensor 

calibration 

Resolve  roll/pitch/ 
heading  for 
installation. 

Perform  opposing  direction  and 
orthogonal  passes  over  baseline. 
Compare  with  nominal  values 
from  standard  installation. 

Prior  to  each 
flight  day. 

+/-  0.02  degrees 

Accomplished.  Standard  roll,  pitch 
and  heading  correction  factors  were 
established  through  calibration 
flights  for  both  sites.  See  Lidar 
Positional  Accuracy  Report,  below. 

4 

Sensor  speed 

Laser  pulse  rate 
between  50-100  kHz. 

Set  laser  pulse  speed  and  the 
altitude  of  the  low  lidar  passes 
depending  on  site  conditions  to 
achieve  highest  possible  point 
density. 

Prior  to  each 
flight  day. 

Achieve  target 
sensor  speed. 

Accomplished.  Laser  pulse  rate  of 

75  kHz  was  used  at  both  sites. 
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5 

Flight  altitude, 
Victorville 

Flight  altitudes  of  450 
and  300  m. 

Establish  and  fly  appropriate 
flight  altitudes  for  the  desired 
lidar  point  densities  and 
orthophoto  pixel  sizes.  Lay  out  a 
series  of  flight  lines  for  high- 
density  lidar  collection  to  be  able 
to  respond  to  site  conditions. 

Each  flight 
line. 

+/-  50  m  from 
planned  flight 
altitudes. 

Accomplished.  Lidar  flight  altitudes 
were  450  and  300  m.  Flight 
altitudes  are  documented  through 
daily  flight  logs. 

6 

Area  coverage 

100%  coverage  for 
each  flight. 

Establish  and  fly  flight  lines  so  as 
to  cover  the  entire  target  area. 

Data  from  each  day’s  flights  will 
be  examined  and  data  gaps  will 
be  filled. 

Each  flight 

100%  coverage. 

Accomplished.  100%  coverage  of 
the  study  area  was  accomplished  for 
each  flight  at  both  sites. 

15%  flightline 
overlap,  50m 
over  area 

boundaries. 

Accomplished.  Flight  line  overlap 
met  specifications  for  all  flights. 

7 

Data  collection 
rate 

Collect  data  for  the 
entire  site  within  the 
established  schedule. 
Reserve  one  additional 
day  for  QA/QC  and  re¬ 
flights 

Establish  and  review  flight  lines 
and  flight  schedule  prior  to  data 
collection. 

NA 

Full  data 
collection  within 
planned  schedule. 

Partially  Accomplished.  Data 
collection  at  Camp  Beale  required 
two  additional  days  due  to  high 
ambient  air  temperature. 

8 

Lidar  point 
density 

Achieve  overall 
densities  of:  5  pts/m^ 
for  the  300  m  flight 
and  3  pts/m^  for  the 

450  m  flight 

Plan  and  accomplish  appropriate 
sensor  speed,  flight  altitude,  and 
air  speed.  Flights  more  than  10% 
below  target  point  densities  will 
be  repeated. 

Each  flight. 

Data  collection 
within  10%  of 
target  densities. 

Accomplished.  Achieved  data 
density  for  the  300  m  flight  was 

13.8  pts/w?  and  13.7  pts/w?  for  the 
450  m  flight. 

9 

Lidar  flight  line 
alignment 

The  450  m  and  300  m 
flights  were  not  to 
exceed  an  intersection 
angle  of  90°,  +/-  10°. 

Appropriate  flight  lines  will  be 
designed  and  flown.  Planned 
flight  lines  will  be  submitted  in 
advance. 

Each  flight 

Flight  lines 
within  10°  of 
orthogonal. 

Accomplished.  No  computed 
intersection  angle  exceeded  the 
tolerance  criteria  of  90°,  +/-  10°. 

10 

Lidar  vertical 
accuracy 

Vertical  accuracy  of 
+/-  15  cm  compared  to 

Steps: 

Each  flight 
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ground  survey. 

1 .  Perform  sensor  calibration  as 
described  above 

As  above 

Accomplished. 

2.  Obtain  ground  elevations  on 
identifiable  points  using 
ground  based  GPS  methods 
(static  and/or  kinematic) 

As  above 

Accomplished.  Control  points  were 
collected  as  described. 

3.  Compare  ground-based  and 
airborne  elevations. 

Meet  or  exceed 
+/1  15  cm  lidar  to 
control  point 
vertical  accuracy. 

Accomplished.  Elevation 
comparisons  were  performed 
between  control  point  elevations  and 
the  interpolated  elevation  of  the 

Lidar  surface  at  that  point.  Results 
were  within  specifications.  See 
Positional  Accuracy  Report. 

11 

Lidar 

horizontal 

accuracy 

Both  Sites:  Horizontal 
accuracy  of  +/-  65  cm 
compared  to  ground 
survey 

Steps: 

Each  flight 

Accomplished.  Positions  of  control 
points  were  obtained  in  the  Lidar 
data  using  intensity  values.  These 
positions  were  compared  to  the 
surveyed  locations  of  these  control 
points.  Horizontal  accuracy  was 
well  within  specification.  See  Lidar 
Positional  Accuracy  Report. 

1 .  Perform  sensor  calibration  as 
described  above 

As  above 

Accomplished. 

2.  Obtain  ground  positions  on 
identifiable  points  using 
ground  based  GPS  methods 
(static  and/or  kinematic) 

As  above 

Accomplished. 

3.  Compare  ground-based  and 
airborne  positions. 

Meet  or  exceed 
+/-  65  cm  lidar  to 
control  point 
horizontal 
accuracy. 

Accomplished. 
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12 

Lidar  data 
integration  - 
flight  lines 

Both  Sites:  Achieve 
flight  line  to  flight  line 
edge  match  of  +/- 
12cm. 

Review  statistics  from  lidar 
processing  software. 

Line  to  line 

Achieve  best 
possible  match 
between 
individual  lidar 
flight  lines. 

Accomplished. 

13 

Lidar  point 
separation 

Both  Sites:  Remove 
100%  of  large  features, 
(trees,  buildings, 
vehicles) 

Operators  remove  non-ground 
laser  returns  through  automated 
separation  routines  followed  by 
hand  cleaning  and  inspection. 

Lidar  data  set 
for  each  flight 

Satisfactory 
visual  inspection 
of  surface  model 
of  the  ground 
surface. 

Accomplished.  Lidar  points  were 
classified  as  ground  and  non-ground 
returns.  Visual  inspection  of  the 
ground  returns  showed  that  all  trees, 
buildings,  fences,  and  other  larger 
features  were  successfully  removed. 

Remove  small  features 
(grass,  low  brush)  to 
the  level  where 
remaining  data  cannot 
distinguish  ground 
from  non-ground 
features. 

Satisfactory 
visual  inspection 
of  surface  model 
of  the  ground 
surface. 

Accomplished.  Inspection  of 
ground  and  non-ground  Lidar  points 
in  conjunction  with  orthophotos 
showed  that  small  brush  and  tall 
grass  was  removed  within 
specification. 

Orthophoto  Data  Collection  and  Processing 

14 

Orthophoto 
area  coverage 

Both  Sites:  100% 
coverage  for  each 
flight. 

Wireframes  of  “raw”  images  are 
compared  to  the  project  boundary 
to  check  for  gaps  or  holes. 

Each  flight 
day  as  part  of 
QA/QC 
checks. 

100%  coverage 
with  sufficient 
image  overlap  for 
ortho¬ 
rectification. 

Accomplished. 

15 

Orthophoto 
flight  altitude/ 
pixel  size 

450  m  (for  10  cm  pixel 
flight) 

Orthophoto  pixel  size  is  directly 
related  to  flight  altitude.  Flight 
lines  are  designed  for  the  desired 
pixel  sizes.  Flight  data  will  be 
examined  during  and  after  each 
flight  and  flight  lines  outside  of 
the  range  will  be  repeated. 

Each  flight 

Achieve  specified 
flight  altitudes 
and  pixel  sizes. 

Accomplished.  10  cm  pixel  sizes 
were  achieved  for  100%  of  the  study 
area. 

16 

Orthophoto 

image 

No  obvious  seams 
between  images  in  the 

Creation  of  an  image  mosaic 
from  individual  small  images  is 

Each 

orthophoto 

Line  features  are 
continuous  with 

Accomplished.  Visual  inspection  of 
the  orthophoto  images  showed  no 
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mosaicing 

final  orthophoto. 

largely  an  operator  controlled 
rather  than  an  automated  process. 

composite 

image 

no  visible 
discontinuity  at 
mosaic  seams. 

obvious  seams. 

17 

Orthophoto 
image  color 
balancing 

No  obvious  color 
imbalances  within  data 
for  each  session. 

Color  balancing  is  an  operator 
controlled  process  based  on 
viewing  the  mosaic  to  identify 
any  areas  of  tonal  imbalance. 

Each 

orthophoto 

composite 

image 

Continuity  of 
tone  such  that 
individual  images 
are  not  visible  in 
mosaic. 

Accomplished.  Visual  inspection  of 
the  orthophoto  images  showed  no 
obvious  color  imbalances. 

18 

Orthophoto 
horizontal 
alignment  to 
lidar 

Lidar  and  orthophotos 
aligned  so  that  target 
features  are  not 
displaced  in  the  two 
data  sets. 

Orthorectification  is  performed 
using  the  lidar  data  and  fiducial 
locations  are  control  data  sources, 
followed  by  operator  adjustment. 

Each 

orthophoto 

composite 

image. 

Orthophotos 
aligned  to  +/-  2 
pixel  widths. 

Accomplished.  Positions  of  control 
targets  were  compared  using  the 
orthophoto  and  lidar  intensity 
values.  Locations  were  within 
specifications. 

19 

Orthophoto 
horizontal 
alignment  to 
fiducials 

Both  Sites: 

Orthophotos  aligned  to 
survey  control  points 
so  that  target  features 
are  not  displaced. 

Orthorectification  is  performed 
using  the  lidar  data  and  fiducial 
locations  are  control  data  sources, 
followed  by  operator  adjustment. 

Each 

orthophoto 

composite 

image. 

Orthophotos 
aligned  to  +/-  3 
pixel  widths. 

Accomplished.  Ortho  image 
positions  were  compared  to  control 
targets  visible  in  the  images.  In 
addition,  lidar  and  orthophoto 
positions  were  compared  for 
building  comers  and  edges  of 
pavement  that  were  visible  in  both 
the  orthophoto  and  lidar  data. 
Positions  were  within  specifications. 
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Table  4-3 

Data  Quality  Metrics,  MRS  Identification  and  Analysis 


MRS  Identification  and  Analysis 

20 

MRS 

identification 

Correctly  identify  all 
previously  identified 

MRS. 

Identify  and  document 

MRS  from  lidar  and 
orthophoto  data  sets. 

Each  lidar  and 
orthophoto  data 
set  and 
combinations. 

Correctly  identify 
all  MRS. 

N/A.  The  initial  CSM  gave  target 
areas  but  not  specific  MRS.  Lidar 
and  orthophotoss  did  locate  one 
MRS  outside  of  the  mapped  target 
areas. 

21 

MRS  false 
alarm  rate 

No  areas  incorrectly 
identified  as  MRS. 

Identify  and  document 

MRS  from  lidar  and 
orthophoto  data  sets. 

Each  lidar  and 
orthophoto  data 
set  and 
combinations. 

No  false  MRS 
identification. 

Not  Yet  Determined.  The  initial 
areas  of  interest  included  some 
areas  of  “potential  MEC  related 
features”  that  were  tentatively 
identified  as  not  munitions  related. 
Many  features  remain  ambiguous. 

22 

MRS 

boundary 

delineation 

Correctly  locate  MRS 
boundaries  to  +/-  15%  of 
ground-truthed  area. 

Identify  and  document 

MRS  boundaries  from 
lidar  and  orthophoto  data 
sets  for  a  selected  set  of 
test  MRS. 

Each  lidar  and 
orthophoto  data 
set  and 
combinations. 

Locate  MRS 
boundaries  within 
metrics. 

Not  Yet  Determined.  Five  MRS 
were  tentatively  identified.  Field 
verification  of  MRS  boundaries 
has  not  yet  been  completed. 

23 

MRS  feature 
identification 

Identify  features 
presenting  as  human- 
made  (anthropogenic)  not 
including  craters  (e.g., 
walls,  berms,  pits,  small 
buildings). 

Lidar  and  photo  data  sets 
will  be  examined  for  linear 
features. 

Each  lidar  and 
orthophoto  data 
set  and 
combinations. 

90%  of  features 
identified  from 
selected  field- 
identified  features. 

Accomplished.  Potential  human- 
made  features  were  readily 
identifiable. 
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Data  management 

24 

Data 

management 

Data  backup  and  storage 
to  achieve  redundancy 
and  security. 

Data  will  be  backed  up  to 
separate  redundant  hard 
drives  or  tape  drives. 

Daily  backup 
during  field  and 
data  processing 
operations. 

Data  security, 
prevention  of  data 
loss. 

Accomplished.  All  data  was  and 
remains  backed  up  to  redundant 
hard  drives. 

25 

Data 

collection 

report 

Standard  flight  reporting 
includes:  calibration  log, 

The  data  collection  report 
is  a  standard  QA/QC 
product. 

Calibration 
report:  each  flight 
day. 

Full  reporting  is  a 
required  part  of 
contract 
performance. 

Accomplished.  Calibration  flights 
were  accomplished  and  standard 
pitch,  roll  and  heading  adjustment 
values  were  calculated  and 
recorded. 

Flight  log,  QA/QC  log, 
and  site  photos. 

Flight  log:  each 
flight. 

Full  reporting. 

Accomplished.  Flight  log  data 
was  recorded  and  delivered. 

QA/QC  log:  each 
flight. 

Full  reporting. 

Accomplished.  Flight  logs  and 
QA/QC  report  were  provided. 

Site  photos: 
whole  project. 

Full  reporting. 

Accomplished.  Site  photos  were 
taken  and  delivered. 

26 

Data 

processing 

report 

Standard  data  processing 
report  includes:  GPS 
control  ties,  accuracy 
verification  report,  and 
QA/QC  report. 

The  data  processing  report 
is  a  standard  QA/QC 
product. 

Each  lidar  and 
orthophoto  data 
set. 

Full  reporting  is  a 
required  part  of 
contract 
performance. 

Accomplished.  GPS  control, 
accuracy  verification  report  and 
QA/QC  reports  were  delivered. 

27 

Metadata 

Metadata  to  accurately 
describe  data  format  and 
processing  steps. 

Data  will  meet  US 
Government  SDS  and  fully 
comply  with  Versar  EDD 
specifications  including 
metadata  standards. 

Each  data 
transfer. 

Metadata  meets 

required 

standards. 

Accomplished.  ESTCP  staff 
stated  that  SDS  standards  would 
not  apply  but  that  metadata  would 
be  required.  Standard  GIS 
metadata  files  were  delivered  with 
all  data. 

28 

QA/QC 

All  data  and  derived 
products  will  be  subject 
to  appropriate  QA/QC 
review. 

Data  processing  will 
follow  the  QA/QC  plan 
described  herein. 

Each  data 
transfer. 

Data  are  valid 
useful  for  the 
intended  purpose 
and  defensible. 

Accomplished.  Each  data 
deliverable  was  independently 
reviewed  by  the  GIS  Lead  and  a 
standard  QA/QC  form  filled  out 
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and  placed  in  the  project  files. 

29 

Data  delivery 

All  data  will  be  delivered 
in  a  timely  and  easy-to- 
transfer  manner. 

Data  deliverables  will  be 
made  using  ftp  where 
possible,  but  in  all  cases 
will  be  followed  up  with 
delivery  on  physical 
media,  primarily  external 
hard  drives. 

Each  data 
transfer. 

Meeting  data 

deliverable 

deadlines. 

Accomplished.  Data  was 
delivered  on  or  before  the  dates 
given  in  the  Demonstration  Plan. 
Interim  data  deliveries  were  made 
by  DVD  or  external  hard  drive. 

Final  data  delivery  was 
accomplished  through  external 
hard  drive. 
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5.0  COST  ASSESSMENT 


5.1  COST  REPORTING 

Table  5-1  presents  actual  costs  for  the  Kirtland,  Victorville  and  Former  Camp  Beale 
demonstration  sites,  and  estimated  costs  for  production  sites  of  two  additional  sizes.  The  figures 
for  the  two  production  sites  are  planning-level  estimates,  assessed  to  be  accurate  within  +/-  20%. 
Per  acre  costs  for  the  Kirtland  site  were  higher  since  four  rather  than  two  lidar  flights  were 
conducted,  and  one  rather  than  two  orthophoto  sets  were  created.  The  Victorville  configuration, 
with  one  lidar/orthophoto  flight  and  one  additional  lidar-only  flight,  is  considered  representative 
for  a  production  site  where  it  is  important  to  detect  both  targets  and  individual  small  features  and 
was  used  at  the  Camp  Beale  site.  The  “50,000  acre  production  site”  estimate  is  based  on  URS’ 
previous  experience  and  interviews  with  industry  sources,  and  the  “1 15,000  acre  production  site” 
estimate  is  based  on  a  cost  proposal  made  by  URS  for  a  site  in  the  western  US  in  Fall,  2005.  All 
figures  are  in  2006  US  dollars  and  costs  were  updated  in  June  2007.  All  projects  listed  can  be 
completed  in  less  than  one  year;  therefore  no  discount  factor  has  been  applied  to  the  figures. 


Table  5-1 

Actual  and  Projected  Costs 


Project  Parameters 

Kirtland 

Victorville 

Former 
Camp  Beale 

50,000  acre 
Production 
Site 

115,000  acre 
Production 
Site 

Project  area  size  (acres) 

5,000 

5,640 

18,000 

50,000 

115,000 

Project  area  size  (hectares) 

1,914 

2,282 

7,284 

19,140 

44,022 

Lidar  flights: 

300  m  (Lidar  only) 

2 

1 

1 

1 

1 

450  m  (Lidar  and  10  cm  pixel 
imagery) 

1 

1 

1 

1 

1 

900  m  (Lidar  and  20  cm  pixel  imagery) 

1 

0 

0 

0 

0 

Total  Lidar  flights 

4 

2 

2 

2 

2 

Total  Lidar  point  density  (pts/m2) 

. 20  __ 

8-10 

8-10 

8-10 

8-10 

Orthophoto  pixel  size  (cm) 

10  and  20 

10 

10 

10 

10 

Costs: 

Fixed  Costs 

Mob/demob 

15,600 

23,100 

21,800 

_ 3_0,000_ 

45,000 

Planning/preparation 

15,000 

9,200 

15,000 

15,000 

20,000 

Project  management 

15,000 

10,000 

25,000 

_ 40,000 

100,000 

Site  work 

0 

0 

0 

0 

0 

Equipment  cost 

0 

0 

0 

0 

0 

Start-up  and  testing 

0 

0 

0 

0 

0 

_ Subtotal  fixed  costs _ 

45,600 

_ 42.300 

. 61,800_ 

_ 85,000 

_ i65£00_. 

FINAL  REPORT  ADDENDUM 
High  Density  Lidar  and  Orthophotography 
in  UXO  Wide  Area  Assessment 
Contract  No.  N44255-02-D-2008 


Section  5.0 
Revision  No.:  2 
Date:  January  2008 
Page  5-2 


Table  5-1  (Continued) 
Actual  and  Projected  Costs 


Project  Parameters 

Kirtland 

Victorville 

Former 
Camp  Beale 

50,000  acre 
Production 
Site 

115,000  acre 
Production 
Site 

Variable  Costs 

Data  acquisition 

39,900 

34,100 

85,300 

160,000 

355,000 

Data  processing 

45,800 

35,200 

102,900 

250,000 

575,000 

Data  analysis  and  GIS  products 

94,300 

30,000 

68,100 

150,000 

220,000 

Data  reporting  and  documentation 

13,600 

8,500 

12,000 

15,000 

25,000 

Materials  and  consumables 

1,500 

1,000 

1,500 

5,000 

10,000 

Other  Direct  Costs 

0 

0 

0 

0 

0 

Subtotal  variable  costs 

195,100 

108,800 

269,800 

580,000 

1,185,000 

Total  project  cost 

240,700 

151,100 

331,679 

665,000 

1,350,000 

Total  per/acre  cost 

48.1 

26.8 

18.43 

13.3 

11.7 

Total  per/hectare  cost 

125.8 

66.2 

45.53 

34.7 

30.7 

5.2  COST  ANALYSIS 

5.2.1  Cost  drivers 

The  major  cost  drivers  for  the  Camp  Beale  site  largely  confirmed  the  findings  from  the  Kirtland 
and  Victorville  sites.  These  were: 

•  Lidar  data  density  required.  For  the  Kirtland  site,  four  lidar  flights  were  conducted; 
two  concurrently  with  digital  imagery  collection,  and  two  lidar-only  flights.  For  the 
Victorville  site,  one  lidar/orthophoto  flight  and  one  lidar-only  flight  were  conducted. 
For  Camp  Beale,  two  lidar  flights  were  conducted. 

•  Orthophoto  data  density  required.  For  the  Kirtland  site,  two  sets  of  digital  images 
were  collected,  and  orthophotos  were  created  at  10  cm  and  20  cm  pixel  sizes.  For  the 
Victorville  site,  only  10  cm  pixel  size  was  collected.  For  the  Victorville  and  Former 
Camp  Beale  sites,  only  10  cm  pixel  orthophotos  were  acquired. 

•  Accuracy  and  precision  requirements.  A  higher  level  of  survey  control  was  needed  at 
the  Camp  Beale  site  than  for  production  sites,  including  collecting  16  control  points, 
10  test  craters  and  4  vertical  control  structures.  For  production  projects,  fewer  survey 
control  points  and  vertical  control  structures  would  likely  be  needed.  However,  the 
cost  of  project  control  is  small  relative  to  that  of  data  acquisition  and  processing,  and 
the  cost  savings  would  be  relatively  minor. 
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•  Site  location  and  logistics.  The  Camp  Beale  site  location  affected  project  costs  both 
positively  and  negatively.  The  site  is  situated  close  enough  to  the  Yuba  Airport  so  as 
to  not  require  establishing  a  fuel  cache  on  site  or  to  require  the  aircraft  to  land. 
Negative  factors  included  the  proximity  of  the  PAWS  radar  installation  which 
interfered  with  the  lidar  equipment  requiring  additional  data  processing  and  the 
excessive  ambient  temperatures  experienced  during  the  data  acquisition  flights  which 
limited  the  flights  to  the  early  morning  hours  and  required  slower  aircraft  speeds. 

In  addition  to  the  cost  drivers  listed  above,  costs  for  production  sites  will  be  affected  by  the 

following  additional  factors: 

•  Site  size.  Larger  sites  achieve  cost  savings  through  amortization  of  fixed  costs  such 
as  mobilization  and  project  planning,  as  well  as  through  increased  efficiency  in  data 
acquisition  and  processing.  This  effect  can  be  seen  in  Table  5-1 

•  Vegetation  conditions.  Highly  vegetated  sites  may  have  higher  costs  due  to  the 
requirement  for  additional  lidar  passes  to  achieve  sufficient  density  of  points  reaching 
the  ground  surface.  Alternatively,  it  may  be  possible  to  achieve  sufficient  vegetation 
penetration  by  specifying  the  use  of  higher  speed  sensor  equipment. 

•  Permitting  and  site  access  constraints.  DoD  sites  with  sensitive,  high-security  areas 
may  have  higher  costs.  However,  such  conditions  would  typically  affect  only  pre¬ 
flight  planning  and  equipment  mobilization  costs  rather  than  data  acquisition, 
processing  and  analysis  costs.  Sites  with  environmental  constraints  do  not  normally 
impose  significantly  higher  costs  for  lidar  and  orthophotography  since  the  airborne 
nature  of  the  technologies  does  not  typically  affect  sensitive  species  or  environments. 

5.2.2  Cost  Sensitivities  and  Additional  Potential  Savings 

Additional  savings  could  be  realized  through  either  of  the  following  methods: 

•  Acquiring  orthophotography  with  a  larger  pixel  size.  The  cost  of  acquiring  and 
processing  orthophotography  rises  dramatically  for  smaller  pixel  sizes,  and  acquiring 
orthophotos  at  20  cm  pixel  size  rather  than  10  cm  would  reduce  the  data  acquisition 
and  processing  costs  by  30  -  35%.  The  utility  of  such  photos  would  be  lower  since 
their  resolution  will  not  allow  discrimination  of  smaller  features. 

•  At  highly  vegetated  sites,  orthophotos  are  inherently  less  useful,  and  orthophotos  with 
larger  pixel  sizes  may  be  acceptable  or  orthophoto  collection  may  be  eliminated 
altogether  if  pre-existing  orthophotography  is  available  and  its  positional  accuracy 
can  be  verified.  However,  at  relatively  open-sky  sites,  site  managers  should  consider 
acquiring  10  cm  pixel  orthophotos.  Experience  during  both  phases  of  the  WAA  Pilot 
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Program  has  shown  that  these  are  significantly  more  useful  than  orthophotos  with 
larger  pixel  sizes. 

Acquiring  lower-density  lidar  data.  Eliminating  the  assumed  second  lidar  flight,  and  thus  only 
collecting  lidar  with  the  10  cm  orthophoto  imagery,  would  reduce  costs  by  25  to  30%.  The 
ability  of  the  resulting  lidar  data  set  to  discriminate  features  would  be  reduced,  however  this 
might  be  appropriate  if  the  lidar  data  was  to  be  used  only  to  discriminate  large  features  such  as 
bombing  targets  or  roads,  rather  than  smaller  features  such  as  craters.  Alternatively,  DoD  could 
specify  use  of  a  faster  lidar  sensor,  which  could  meet  lidar  data  density  requirements  from  a 
single  pass. 

Some  additional  cost  savings  could  potentially  be  achieved  by  establishing  Service-  or  DoD- 
wide  standards  for  data  acquisition,  GIS  data  product  creation,  data  delivery  formats,  and  project 
reporting. 

5.3  COST  COMPARISON 


Cost  comparisons  with  the  other  innovative  technologies  demonstrated  as  part  of  the  ESTCP 
WAA  Pilot  Program  will  be  made  in  the  Final  Report  for  the  WAA  Pilot  Program. 


FINAL  REPORT  ADDENDUM 
High  Density  Lidar  and  Orthophotography 
in  UXO  Wide  Area  Assessment 
Contract  No.  N44255-02-D-2008 


Section  6.0 
Revision  No.:  2 
Date:  January  2008 
Page  6-1 


6.0  IMPLEMENTATION  ISSUES 

6.1  ENVIRONMENTAL  CHECKLIST 

No  environmental  regulations  applied  to  the  demonstration  and  no  permits  were  required. 

6.2  OTHER  REGULATORY  ISSUES 

Both  lidar  and  orthophotography  are  in  wide  commercial  use.  Within  the  United  States,  no 
regulatory  restrictions  are  known  that  would  impede  the  wide  use  of  either  technology  at  DoD 
sites.  Outside  of  the  United  States,  use  of  advanced  IMU  equipment  may  be  restricted  in  certain 
countries.  The  IMU  used  in  lidar  systems  is  military  dual  use  technology  and  international  use 
requires  a  permit  under  the  International  Traffic  in  Arms  Regulations  (22  CFR  120-130). 
Additionally,  some  countries  impose  a  variety  of  restrictions  on  the  acquisition,  processing  and 
subsequent  use  of  lidar  and  orthophoto  data  collected  within  their  borders,  particularly  in  border 
or  military-use  areas.  Potential  users  of  lidar  in  such  situations  should  investigate  such 
restrictions  as  part  of  project  planning. 

6.3  END-USER  ISSUES 

Both  lidar  and  orthophotos  are  in  wide  commercial  use  and  do  not  face  substantial  end-user 


issues. 
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